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Executive Summary 


In 2006 4-H National Headquarters at the U.S. Department of Agriculture and National 
4-H Council introduced a new initiative aimed at increasing the number and quality of science, 
engineering, and technology programs that 4-H offers around the country, and increasing the 
number of youth involved in these programs. By engaging youth in informal science educational 
opportunities through the 4-H Science Initiative, the organization hopes to increase: science 
interest and literacy among youth, the number of youth pursuing post-secondary education in 
scientific fields, and the number of youth pursuing science-related careers. With the support of 
the Noyce Foundation, National 4-H Council has contracted with Policy Studies Associates 
(PSA) to evaluate the implementation of this new initiative. 


Annually, nearly 6 million youth participate in 4-H, which is implemented by 106 Land- 
Grant Universities and Colleges (LGUs) in more than 3,000 counties as a part of the Cooperative 
Extension System. National leadership is provided by 4-H National Headquarters at the National 
Institute for Food and Agriculture, USDA, and National 4-H Council, which is the national 
nonprofit partner of 4-H and the Cooperative Extension System. National 4-H Council focuses 
on fundraising, branding, communications, and legal and fiduciary support to 4-H programs. 


The National 4-H Science Evaluation Design Team created a logic model for the 
initiative that outlines the short- and long-term goals of the initiative. According to the logic 
model, programs should be designed with the following short-term youth outcomes in mind: 


Increased awareness of science 

Improved scientific skills (scientific methods) and knowledge (content areas) 
Increased awareness of opportunities to contribute to society using science 
Increased life skills 


4-H selected a sample of LGUs to participate in Year 2 of the evaluation. The LGUs that 
participated in 2011 data collection include: 


The University of California, Davis 
Cornell University 

The University of Delaware 

Iowa State University 

The Ohio State University 

North Carolina A&T University 
Texas A&M University 


This year’s evaluation had three key objectives: (1) to gather program-level information 
about the implementation of 4-H science programs from science educators, (2) to collect 
information from participants about their attitudes toward science and their opinions of their 4-H 
Science programs, and (3) to look for any associations between youth-level data and the 
characteristics of the 4-H Science programs in which youth are enrolled. 


In order to meet these goals, we conducted a survey of science educators leading 4-H 
Science programs, and a survey of youth participants. We administered the science educator 
survey in spring 2011 had an overall response rate of 62 percent. We administered youth 
surveys in a random sample of programs in late May and June 2011. We received youth surveys 
from 19 of the 21 programs in the sample, for a program-level response rate of 91 percent.' We 
requested that programs administer surveys to all youth between the ages of 9 and 18 who were 
present on the designated survey administration day.” The overall youth-level response rate was 
66 percent. For the 19 programs that returned youth surveys, the response rate was 76 percent. 


Implementation of Science Programming 


Enrollment and attendance. Across science programs, enrollment ranged from two to 
3,399 youth, with a median of 26 participants enrolled in each program. Reported attendance at 
program meetings ranged from a low of one youth to a high of 550 youth, with a median of 15 
youth attending each program meeting. 


Content areas. 4-H Science programs cover a range of curricular areas, including 
traditional 4-H content areas that have been revised to meet the criteria outlined in the Science 
Checklist, as well as content areas new to 4-H such as rocketry and robotics. Large and small 
animal science (36 percent), environmental science (23 percent), and environmental stewardship 
(14 percent) were among the content areas most frequently addressed in Science programs. 
While programs in content areas such as energy and new technologies often garner much 
attention, these areas actually represent a minority of 4-H Science program areas (Exhibit E1). 


' The majority of the programs we contacted in New York and Delaware did not return youth surveys. Because 
including the small number of surveys we received would not be representative of those states, we excluded both 
states from our sample. 

* One camp did not have the capacity to survey all youth due to the large number of participants. To alleviate the 
data collection burden on this program, we asked that they only survey youth in its two oldest age groups. 
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Exhibit E1 
Content Areas of 4-H Science Programs (n=183) 


Content Area Programs 
Environmental Science Pi a8 | 
Animal Science-Large Animal 19 

Animal Science-Small Animal 17 
Environmental Stewardship 14 
Engineering 12 

Physical Sciences 10 

Food Science 

Aerospace and Rocketry 

Robotics 

Gardening 

Plant Science 

Earth Science 

Family and Consumer Sciences 

Technology 


Veterinary Science 


Weather and Climate 

Horticulture 

Computer Technology 

Geospatial Technology (GPS/GIS) 


Exhibit reads: Twenty-three percent 4-H Science educators 
reported that their program focused on environmental science. 


Curricula. The majority of science educators said that they used an established 
curriculum to guide their program. Eighty-four percent of all programs reported using curricula 
developed by 4-H or by an external organization. Among the programs that use established 
curricula, 77 percent reported using a curriculum developed by 4-H. Specifically, eighteen 
percent reported using 4-H Animal Science curricula, followed by eight percent that use Acres of 
Adventure (Exhibit E2). 
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Exhibit E2 
Use of 4-H Curricula (n=183) 


There's No New Water 


Robotics 4% 
5% 


Exploring Your Environment 
6% 


Power of Wind Other 4-H Curricula 
6% 30% 
Acres of Adventure 
8% 


Animal Science 


4H 
18% Wedonotusea 


Science Curriculum 
23% 


Exhibit reads: Eighteen percent of science leaders that use curricula reported using 4-H Animal Science curricula in 
their program. Thirty percent of science leaders use a 4-H curricula other than those listed in the survey, and 23 
percent of science leaders did not use 4-H curricula in their programs. 


Programming objectives. When asked to list the primary objectives of their science 
programs, educators largely reflected 4-H’s expectations with an overwhelming majority saying 
that providing hands-on experiential activities (93 percent), encouraging youth to develop an 
interest in science (79 percent), or helping youth develop pro-social and interpersonal skills (67 
percent) were major objectives of their programs. These data reflect recent research on best 
practices for both science education and youth development programming. Successful out-of- 
school programs not only promote learning content and skills, but provide youth with 
opportunities to develop positive relationships among themselves and between youth and 
program staff that are different from the relationships they build during the school day (Eccles & 
Gootman, 2002). 


Activity offerings. Eighty-two percent of 4-H Science programs engage youth in 
activities that encourage them to work in teams or small groups—allowing youth to develop 
interpersonal skills as they master science content—and more than half of programs (61 percent) 
integrate activities that require youth to build or construct models to demonstrate scientific 
phenomena. Youth also have opportunities to practice science-related skills, including gathering 
data or information (52 percent), developing questions and conducting research (45 percent), and 
analyzing and interpreting data (43 percent). 
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Planning and intentionality. Eighty-one percent of educators reported using lesson 
plans to guide activities at some point. Of this group, less than half reported using lesson plans 
consistently for all program meetings (42 percent). 


Characteristics of science educators. The data we collected on science educators 
represents both volunteers and paid staff members who lead 4-H Science programs. Many 
science educators leading programs held degrees in science-related fields and/or had previous 
experience working with youth. Eighty percent of educators reported having attained an 
undergraduate or graduate degree (Exhibit 9). Among educators who received a college or 
graduate degree, 43 percent majored in a science-related field. Thirty-four percent of 
respondents had previous volunteer or work experience in youth programs with a science focus. 


Training and support from 4-H. A majority of science educators reported that they 
spoke with county and state 4-H representatives at least once per year (89 percent). Forty-three 
percent of science educators reported speaking with representatives several times per year, and 
20 percent of science educators spoke with state and county representatives a few times per 
week. Ninety percent of educators agreed that the training and support they received in youth 
development was adequate, and 81 percent responded similarly about the trainings offered to 
support science content delivery. 


Challenges. Some programs encountered challenges to implementing programming. A 
lack of funding to purchase science-related curricula (39 percent) and limited access to 
appropriate physical space (35 percent) were among the most frequently described challenges. 
Lack of volunteers with science skills or knowledge also created a challenge for science educators 
(22 percent). 


4-H Science Participants 


In Year 2 of the evaluation, we used a sampling procedure that drew a random selection 
of programs, surveyed all participants in the sampled programs, and achieved high response 
rates. We therefore believe these youth survey data are representative of 4-H Science 
participants in the seven evaluation states. 


Demographics. We received surveys from a total of 486 youth ranging from 9 to 18 
years of age. Sixty-nine percent of youth were between 9 and 12 years of age, and 31 percent 
were between ages 13 and 18. Females comprised the majority of the youth sample (59 percent). 
Sixty-one percent of youth survey respondents reported their race or ethnicity as white, 25 
percent as African American, 12 percent as other, 8 percent as Hispanic/Latino, 4 percent as 
Native American, 3 percent as Asian-American, and 1 percent as Native Hawaiian/Other Pacific 
Islander. (Youth could select more than one race/ethnicity.) 


Youth participation in 4-H. One of the goals of the 4-H Science Initiative is to attract 
new youth to 4-H through science programming. Many of the youth surveyed in this evaluation 
were indeed new to 4-H: 44 percent of respondents said that this was their first year. 


Extent of weekly involvement. Youth spend varying amounts of time each week in their 
current 4-H Science programs. Roughly one-quarter of youth reported that they spend one or 
hour or less each week on their project (28 percent), and a quarter of youth spend between one 
and three hours each week (25 percent). Nearly half of respondents (47 percent) spend over 
three hours per week in their program. These variations may be due to different levels of youth 
attendance at programs, but may also reflect the variety of ways that 4-H science programming is 
delivered: in after-school settings, during school, in clubs during the week or on the weekends, 
or in a camp. 


Attitudes toward science. In order to begin to build a broad set of data on interest in 
science among youth enrolled in informal science education programs, the Noyce Foundation 
worked with Dr. Cary Sneider of Portland State University, Dr. Gil Noam of Harvard University, 
and Foundation grantees to develop a set of youth survey items that measure youths’ enthusiasm 
for science. 


Youth responses to these items suggest that youth who attend 4-H Science programs are 
enthusiastic about science and eager to participate in science activities, especially in informal 
settings. Most participants reported that they enjoy visiting science museums or zoos, like to see 
how things are made, and like to participate in science projects. Youth were less likely to report 
that they enjoyed reading print materials about science; however, this may simply reflect youths’ 
increasing use of electronic media and decreasing use of print materials. 


National Assessment of Education Progress (NAEP) science assessment comparisons. 
In order to compare 4-H youths’ attitudes toward science against attitudes of a representative 
sample of youth, we included a set of items on the survey that were taken from the NAEP 
Science assessments from 2005 and 2009.** The NAEP Science assessment is administered to a 
nationally representative sample of fourth-, eighth-, and twelfth-grade students roughly every 
four years and includes a set of items designed to measure respondents’ interest in science. 


Overall, youth in 4-H Science programming were more enthusiastic about science than 
their peers in the NAEP sample. While the differences between the groups are apparent, the data 
do not tell us why 4-H youth are more enthusiastic about science, only that there is an association 
between enrollment in 4-H Science and youth reports of their interest in science. This 
association may exist because 4-H Science programs attract youth who have a pre-existing 
interest in science, and therefore give more enthusiastic responses to items designed to measure 
their engagement in science. 


Educational and career aspirations. Youth in 4-H Science programming reported 
having high educational aspirations. When asked how far they want to go in school, half of 
youth surveyed (52 percent) want to finish college, while an additional 37 percent reported that 
they want to get more education after completing a college degree. When we asked youth if they 


> The NAEP science assessment did not include all science attitudinal items in both the 2005 and 2009 assessments. 
For information on which NAEP items were asked in which administration years, please refer to Appendix D. 

* Some of the NAEP items were also part of the Noyce Enthusiasm for Science item set; please see Appendix B for 
more information. 
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wanted to pursue a science-related career after graduating from high school, 54 percent of youth 
agreed or strongly agreed. 


Science skills. The vast majority (87 percent) of youth ages 9-12 reported that they can 
make a chart or picture to show information, while 86 percent said that they can do an 
experiment to answer a question. We asked youth participants ages 13 to 18 a similar set of 
questions about their mastery of certain science process skills. Most youth reported that they can 
always or usually: use the results of their investigation to answer the questions they asked (83 
percent), ask a question that can be answered by collecting data (80 percent), or record data 
accurately (77 percent). 


Program climate and benefits. When asked to select the three things they liked best about 
their science program, youth were most enthusiastic about the relationships they have there: 65 
percent of youth said that the opportunity to spend time with their friends was their favorite part of 
their program, and 35 percent said that the fact that adults were caring and kind was their favorite 
aspect. In addition to the relationships that youth develop in their programs, youth were 
enthusiastic about the science learning opportunities that their programs provide: 52 percent of 
youth said that getting to do hands-on science activities and projects is one of their favorite things 
about their program. 


Analysis of youth survey responses based on youth characteristics. Youth who reported 
attending their 4-H Science program more often and for more hours each week were more likely to 
rate their own life skills highly, to participate in more community science activities, and to have 
higher overall opinions of their science programs. 


Analyses of youth survey responses revealed that boys were slightly more enthusiastic about 
participating in science activities than girls. However, girls were more likely to rate their program 
climates highly, and to give better evaluations of their own life skills. 


Youth belonging to racial and ethnic groups historically underrepresented in the science 
fields were less likely than youth from well-represented groups to report an interest in pursuing a 
science career, and less likely to rate their own science and life skills highly. Youth from under- 
represented groups, on average, rated their program’s climate and the benefits they received from 
attending lower than did youth from well-represented groups. 


Recommendations 


7 Because some science educators reported facing challenges related to the 
affordability of science curricula, 4-H should continue to facilitate the exchange 
of curricular resources among states and counties, perhaps by continuing to 
promote online networks where educators can identify affordable curricula. 


a Educators also noted that finding adequate space in which to host their 4-H 


Science programs was a challenge. 4-H may therefore want to collect information 
regarding best practices and field-tested tips to help educators identify and secure 


Vii 


space for their programs, and then share this information with the state- and 
county-level staff who can advise their educators. 


In order to support and develop girls and youth from racial and ethnic groups that 
are underrepresented in the science fields, 4-H should continue to seek out best 
practices for recruiting, engaging, and supporting these youth, and to share this 
information among state- and county-level leadership. 
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Overview of the Initiative 


In 2006, 4-H National Headquarters, U.S. Department of Agriculture, and National 4-H 
Council introduced a new initiative aimed at increasing the number and quality of science, 
engineering, and technology programs that 4-H offers around the country, and increasing the 
number of youth involved in these programs. By engaging youth in informal science educational 
opportunities through the 4-H Science Initiative, the organization hopes to increase: science 
interest and literacy among youth, the number of youth pursuing post-secondary education in 
scientific fields, and the number of youth pursuing science-related careers. With the support of 
the Noyce Foundation, National 4-H Council has contracted with Policy Studies Associates 
(PSA) to evaluate the implementation of this new initiative. 


Annually, nearly 6 million youth participate in 4-H, which is implemented by 106 Land- 
Grant Universities and Colleges (LGUs) in more than 3,000 counties as a part of the Cooperative 
Extension System. National leadership is provided by 4-H National Headquarters at the National 
Institute for Food and Agriculture, USDA, and National 4-H Council, which is the national 
nonprofit partner of 4-H and the Cooperative Extension System. National 4-H Council focuses 
on fundraising, branding, communications, and legal and fiduciary support to 4-H programs. 


4-H has set an enrollment goal: by the end of 2013, | million youth who have never 
before been in 4-H will enroll in 4-H Science programs. In order to meet this goal, 4-H has 
sought since 2006 to increase its capacity and infrastructure for providing 4-H Science 
programming. As part of this effort, 4-H formed the National 4-H Science Leadership Team, 
which consists of national, state and county-level 4-H professionals. In addition, Science 
Liaisons have been appointed at LGUs around the country to help implement programs and to 
recruit youth into these programs. 


The National 4-H Science Evaluation Design Team created a logic model for the 
initiative that outlines the short- and long-term goals of the initiative. According to the logic 
model, programs should be designed with the following short-term youth outcomes in mind: 


7 Increased awareness of science 

rT] Improved scientific skills (scientific methods) and knowledge (content areas) 
7 Increased awareness of opportunities to contribute to society using science 

a Increased life skills 


Evaluation Methodology 


The goals of this evaluation are to describe characteristics of 4-H Science programs in 
selected states, and to understand the attitudes youth in these programs hold towards science. 
This report is based on the results of two surveys: a survey of science educators leading 4-H 
Science programs, and a survey of youth in a sample of programs. 
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This section of the report details the approach the evaluation team used to document and 
assess 4-H Science programs in the states selected to participate in this second year of the 
evaluation. 


Participating Land Grant Universities 


4-H selected a sample of nine LGUs to participate in Year 2 of the evaluation. However, 
two LGUs were removed from the sample: one LGU reported that it did not have staff available 
to facilitate data collection, and another LGU did not participate because the evaluation team did 
not receive IRB approval to conduct research there until the evaluation’s data collection window 
had passed. The seven LGUs that participated in 2011 data collection include: 


The University of California, Davis 
Cornell University 

The University of Delaware 

Iowa State University 

The Ohio State University 

North Carolina A&T University 
Texas A&M University 


Identifying 4-H Science Programs 


While many 4-H clubs, camps, after-school programs, and school enrichment programs 
address content areas that are related to science—for example, embryology or food and 
consumer science—not all science-related 4-H programming is considered to be part of the 4-H 
Science Initiative. 4-H has therefore developed tools for state and county staff to use in 
assessing which of their science-related programs should be considered a part of 4-H Science. 


4-H Science Checklist. At the onset of the 4-H Science Initiative, 4-H developed a set of 
expectations, referred to as the 4-H Science Checklist, which outlines what characteristics 
programs must meet to be considered part of the Science Initiative’s national evaluation. The 4- 
H Science Checklist requires that programs: (1) be based on National Science Education 
Standards, (2) develop participants’ science-related skills and abilities, (3) use positive youth 
development practices, (4) are led by staff who are well-trained in youth development and 
appropriate content, (5) use an experiential approach to learning, (6) foster creativity and 
curiosity among participants, and (7) address outcomes on the 4-H Science Logic Model.° 4-H 
programming that meets the Checklist criteria is labeled as “Science Ready.” 


ACCESS 4-H. Currently, information on which 4-H programs qualify as Science Ready 
is tracked in different ways by different states. However, 4-H has implemented the use of a 
database called ACCESS 4-H that is designed to track information about 4-H programming and 
participants. Those LGUs that adopt the system are able to enter and track data about all of their 


° The 4-H Science Checklist can be found in Appendix A. 
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4-H programs, including a program’s Science Ready status. Once this information is captured in 
ACCESS, 4-H will be able to readily identify programs that qualify as Science Ready, and 
analyze implementation and participation data from those programs. Although we did not use 
ACCESS 4-H data in Year 2, we expect to utilize data from the system for evaluation activities 
in Year 3. 


Requesting Science Ready program information from state leadership. In the future, 
states that use ACCESS 4-H will be able to easily download information on which of their 
science programs meet the standards set forth in the Checklist. At this point, in order to establish 
a group of Science Ready programs to participate in Year 2 evaluation activities, we asked state- 
level leadership to identify Science Ready programs in their states. Using this information, we 
contacted the lead staff members or volunteers in these programs and began administration of the 
evaluation’s two Year 2 surveys: the survey of science educators, and the Youth Engagement, 
Attitudes, and Knowledge survey. 


Survey of 4-H Science Educators 


In order to learn more about the goals of 4-H Science programs and the types of activities 
they feature, we conducted a survey of 4-H Science educators. Using the lists of 4-H Science 
Ready programs generated by state-level leadership, we contacted the extension educators, staff 
members, and volunteers who serve as the leader of a 4-H Science Ready program and asked 
them to complete a survey. The survey asked leaders to verify that their program was science- 
focused, and to provide information about its goals, its daily activities, and the content areas it 
addressed. As shown in Exhibit 1, the overall response rate for the survey of 4-H Science 
educators was 62 percent; individual states’ response rates ranged from a low of 35 percent in 
Delaware to a high of 100 percent in North Carolina. 


Exhibit 1 
Science Educator Survey Response Rates (n=182) 


Response Rate 


The Ohio State University (n=35) 
Texas A&M University (n=11) 


Exhibit reads: One hundred percent of 4-H educators leading Science 
Ready programs in North Carolina responded to the survey. 


In Year 1 of the evaluation we surveyed county-level 4-H staff and asked them to provide 
information about the Science Ready programs in their counties. Because of a low response rate 
to that survey, the programs we selected from that pool to administer the participant survey were 
not representative of the 4-H Science Initiative overall. In Year 2 we worked with a smaller 
number of LGUs, and put renewed emphasis on achieving high response rates in our follow-up 
efforts in order to increase the representativeness of our final datasets. 


Survey of Youth in Science Ready Programming 


Youth survey design. As part of the evaluation of Science programs, PSA worked with 
the 4-H Science Instrument Design Team: Melissa Cater, Mary Arnold, Lisa Bouillion Diaz, 
Katherine Heck, June Mead, Beverly Spears, Ben Silliman, and Maureen Mulroy, and with Jill 
Walahoski and Suzanne LeMenestrel to develop the 4-H Science Youth Engagement, Attitudes 
and Knowledge (YEAK) survey in Year | of the evaluation. PSA first administered the survey 
on a national scale in winter 2009 (Year 1), and conducted a second national administration in 
late spring/early summer 2011 (Year 2). The YEAK survey serves to address the following 
questions about the youth in the survey sample: 


a What are the characteristics of the youth involved in Science Ready 
programming? 
7 What are participants’ attitudes toward the scientific fields? Do participants 


aspire to pursue career opportunities in science-related fields? 
7 What level of education do participants want to achieve? 


a To what extent are participants engaged in both formal and informal science 
learning? Do participants pursue science leadership opportunities? 


7 What science-related skills, abilities, and knowledge do participants have? 


7 What factors, if any, are associated with participants’ engagement, attitudes, and 
knowledge of science? 


Noyce Enthusiasm for Science survey items. In July 2010, The Noyce Foundation 
convened a group of its out-of-school time grantees to discuss the potential of streamlining their 
evaluation approaches in order to develop a common set of evaluative measures across the 
grantee group. Led by Dr. Cary Sneider and Dr. Gil Noam, the group identified a set of youth 
and staff survey items—referred to as the Noyce Enthusiasm for Science items—which all 
grantees committed to administering during their 2010-11 data collection efforts.° 


Sampling approach. In order to generate youth-level data that would be representative 
of all 4-H Science youth in the states participating in the evaluation, we developed a program- 


° The Noyce Enthusiasm for Science survey items are flagged in the youth and science educator surveys, which can 
be found in Appendix B. 
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level random sampling process for our youth survey administration. The objective of 
administering surveys in a random sample of programs was to ensure that the programs we 
selected were representative of the larger population of 4-H Science programs in those states. By 
randomly selecting programs and not simply asking 4-H leadership to name programs for us to 
survey, we ensured that typical 4-H Science Ready programs were included, and that our sample 
was not limited to high-visibility or high-functioning programs. 


Using the sampling frame created through our administration of the 4-H Science educator 
survey, we selected a sample of Science Ready programs operating in May and June of 2011 in 
which to administer the YEAK survey to participants. We generated a random sample of five 
programs operating in May or June 2011 in each state in order to ensure that our sample would 
be representative. For those LGUs that had five or fewer Science Ready programs operating in 
May or June, we included all programs in our sample. 


Program-level response rates. We administered youth surveys in late May and June 
2011. We received surveys from 19 of the 21 programs in the sample, for a program-level 
response rate of 91 percent.’ 


Youth-level sample. We requested that programs administer surveys to all youth 
between the ages of 9 and 18 who were present on the designated survey administration day.* In 
order to ensure that our sample would be representative, we emphasized the importance of 
administering surveys to all youth, regardless of their science abilities or engagement in the 
program. The overall youth-level response rate was 66 percent. For the 19 programs that 
returned youth surveys, the response rate was 76 percent. 


Exhibit 2 
Program- and Youth-Level Response Rates 


Number of | Programs | Number of | Completed | Youth-level 
programs returning youth youth response rate 
Sa eo a a (in re ) 


The | The University of California, Davis | of California, Davis 


echo senwesy [+ | [aa ane 
Pom cari asTunesy | «fe [|e | 
Pearastwey fs se 


Exhibit reads: fe the five University of Pa! Davis 4-H eral programs that were sent surveys, five 
returned surveys. Of the 94 youth who were surveyed in California, 59 completed surveys, for a response 
rate of 63 percent. 


Generalizeability of data. The evaluation team’s efforts to identify Science Ready 
programming by working with state leadership and then surveying Science educators helped to 


7 The majority of the programs we contacted in New York and Delaware did not return youth surveys; because their 
data would not have been representative, we excluded both states’ data from our analyses. 

* One camp did not have the capacity to survey all youth due to the large number of participants. To alleviate the 
data collection burden on this program, we asked that they only survey youth in its two oldest age groups. 
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ensure that we developed a robust understanding of the Science Ready programming offered in 
each study state. While the response rate to the Science educator survey was slightly lower than 
expected, this 62 percent response rate marked a great improvement over Year 1. 


By drawing a random sample of programs to survey, and then achieving a youth-level 
response rate of 76 percent in programs that returned surveys, we believe that our data are 
largely representative and that our findings are generalizeable to the larger 4-H Science 
population in the states participating in the evaluation.” 


Statistical Tests Employed 


This report provides both descriptive statistics and analyses of the relationships between 
various items from the youth survey. We used a threshold of p<0.05 to identify statistically 
significant findings, and then computed an effect size to measure the magnitude or strength of 
those findings. Conventions for educational research suggest that effect size values between 
0.10 and 0.20 indicate a “small but meaningful” association, between 0.21 and 0.50 an 
“important” association, and 0.51 or higher an “impressive” association (Cohen, 1988; Lipsey, 
1990). This report focuses on findings with an effect size of at least 0.20; comparisons or 
associations below this threshold were considered too weak to warrant reporting. 


Organization of the Report 


The first section of this report describes findings from the survey of all 4-H Science 
educators in the seven states participating in the evaluation, the second section presents data 
from the YEAK survey, and the final sections summarize this evaluation’s findings and present 
recommendations. 


Implementation of Science Programming 


In this section we review findings from the survey of 4-H Science educators in all seven 
states originally selected to be part of this year’s evaluation. We asked educators to provide 
information ranging from enrollment to program activity offerings to challenges they face in 
implementing their science programs. 


Overview of Program Characteristics 


Recruitment. One of 4-H’s goals for the Science Initiative is to attract youth who have 
never before been involved in the organization by sparking their interest through participation in 
a science program. In order to attract participants, science educators reported using various 


” While there is not a specific response rate that, if achieved, guarantees the risk of non-response error, a response 
rate near 70 percent is typically considered acceptable. 
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methods to share information about their programs and encourage youth to enroll. While more 
than half of the science educators reported that youth typically find out about their science 
programs through word-of-mouth (53 percent), many also employed various efforts to get word 
of their programs out to youth. Forty-eight percent reported that youth learn about their 
programs through their schools, and 45 percent of programs reported that participants also learn 
about their programs through participating in other 4-H programs. In addition to these methods, 
educators also said that they advertise their programs through the 4-H website (29 percent), and 
by distributing flyers throughout their communities (28 percent). 


Programming type. Programming occurred at varying times depending on the delivery 
mode and the nature of the science program, with after-school being the most popular meeting 
time. Educators could select more than one meeting time in their response, and just under half of 
educators indicated that their programs met during multiple meeting times. After-school was the 
most popular meeting time, with more than half of science educators (54 percent) indicating that 
their programs met after school. Summer was also a popular meeting time for programs, with 41 
percent of educators indicating that their programs occurred during the summer. Programs also 
met on weekends (33 percent) and during the school day (27 percent). 


Enrollment and attendance. Across science programs, enrollment ranged from two to 
3,399 youth, with a median of 26 participants enrolled in each program. Reported attendance at 
program meetings ranged from a low of one youth to a high of 550 youth, with a median of 15 
youth attending each program meeting. These attendance figures include data from both small 
clubs and bigger programs in which large numbers of youth might participate for one day at a 
large-scale, state-wide event, or as part of a school enrichment program that takes place in 
multiple classrooms and schools. 


Youth served. Science programs served youth between the ages of 5 and 18. About two- 
thirds of programs served youth in the middle of this age range (ages 9 to 12), as seen in Exhibit 
3: 


Exhibit 3 
Ages Served by 4-H Science Programs (n=180) 


Percent of Programs 
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Ages of Youth Served by Science Programs 


Exhibit reads: Twenty-six percent of science educators reported that their 4-H Science program serves five-year-olds. 
(Educators could select multiple ages.) 


Content areas. 4-H Science programs cover a range of curricular areas, including 
traditional 4-H content areas that have been revised to meet the criteria outlined in the 4-H 
Science Checklist, as well as content areas new to 4-H such as rocketry and robotics. Large and 
small animal science (36 percent), environmental science (23 percent), and environmental 
stewardship (14 percent) were among the content areas most frequently addressed in Science 
programs (Exhibit 4). 


Exhibit 4 
Content Areas of 4-H Science Programs (n=183) 
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Exhibit reads: —— —— of 4-H Science educators 
reported that their program focused on environmental science. 


Curricula. The majority of science educators said that they used an established 
curriculum to guide their program. Eighty-four percent of all programs reported using curricula 
developed by 4-H or by an external organization. Among the programs that use established 
curricula, 77 percent reported using a curriculum developed by 4-H. Specifically, 18 percent 
reported using 4-H Animal Science curricula, followed by 8 percent that use Acres of Adventure 
(Exhibit 5). Thirty percent of these programs reported using other 4-H curricula, including 
Youth Experiences in Science, food science curricula, and rocketry curricula. 


Exhibit 5 
Use of 4-H Curricula (n=183) 


There's No New Water 
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Exploring Your Environment 
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Exhibit reads: Eighteen percent of science leaders that use curricula reported using 4-H Animal Science curricula in 
their program. Thirty percent of science leaders use a 4-H curricula other than those listed in the survey, and 23 
percent of science leaders did not use 4-H curricula in their programs. 


In addition to the programs that use a 4-H curriculum, 34 percent of programs said that 
they used curricula developed by organizations other than 4-H. These organizations included 
universities, state and federal agencies, and First Lego League. 


Programming Objectives and Activities 


Current research on out-of-school time programming suggests that these programs should 
be well organized, and feature engaging activities that have specific and clear learning goals for 
youth (Eccles & Gootman, 2002; McLaughlin, 2000; Noam, 2008; Vandell et al., 2006). Recent 
trends in school-day science education priorities also acknowledge the importance of active, 
experiential learning. In July 2011, the National Research Council published a Conceptual 
Framework for K-12 Science Education Standards. In the framework, the committee suggests 
that K-12 education can learn from best practices in informal science education by offering 
hands-on learning opportunities (NRC, 2011). Providing youth with the opportunity to practice 
science, the committee states, is critical to building scientific understanding: “Science is not just 
a body of knowledge that reflects current understanding of the world; it is also a set of practices 
used to establish, extend, and refine that knowledge. Both elements—knowledge and practice— 
are essential.” 
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Successful out-of-school programs not only promote learning content and skills but also 
provide youth with opportunities to develop positive relationships among themselves and 
between youth and program staff that are different from the relationships they build during the 
school day (Eccles & Gootman, 2002). Durlak and Weissberg’s meta analysis of youth program 
evaluations (2007) also suggests that by employing staff who show positive affect toward youth 
and who engage personally with youth, programs can actually improve youth social and 
developmental outcomes. 


In line with research on youth development and informal science programming, the 4-H 
Science Checklist outlines 4-H’s expectation that 4-H Science programs provide skill-driven, 
experiential learning opportunities, as well as a positive youth development context in which 
participants are able to develop relationships with other youth and with program staff and 
volunteers. 


In order to better understand the nature of the activities offered as part of 4-H Science 
programs, the evaluation team asked educators to report on their program’s goals, and the 
frequency with which youth participate in various activities and program opportunities. 


Programming objectives. When asked to list the primary objectives of their science 
programs, educators largely reflected 4-H’s expectations with an overwhelming majority saying 
that: providing hands-on experiential activities (94 percent), encouraging youth to develop an 
interest in science (79 percent), or helping youth develop pro-social and interpersonal skills (67 
percent) were major objectives of their programs (Exhibit 6). 
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Exhibit 6 
Major Program Objectives (n=184) 


Provide hands-on, experiential learning 
activities 


Encourage youth to develop an interest in 


Help youth develop pro-social and 
interpersonal skills 


Help youth become aware of the roles science 
plays in their everyday lives 


Teach youth about a specific science content 


Help youth practice science-related skills 


Help youth understand how science can help 
them to improve their communities 


Use national science education standards to 
inform and guide activities 


Percent 


Exhibit reads: Ninety-four percent of educators reported that a goal of their program is to provide hands-on, 
experiential learning opportunities. 


More than half of all science educators also reported that their programs aim to: help 
youth become aware of the roles science plays in their everyday lives (62 percent), teach youth 
about a specific science content area (61 percent), and help youth practice science-related skills 
(59 percent). 


Activity offerings. When asked to report on the frequency with which youth participated 
in certain types of learning activities, science educators reported that youth not only practiced 
science skills such as gathering and interpreting data, but also had many opportunities to work in 
groups and use teamwork to solve problems. This focus on communication and interpersonal 
skills reflects the National Research Council’s Committee on Conceptual Framework for K-12 
Science Education Standards’ suggestion that science education reflect that “science is 
fundamentally a social enterprise, and scientific knowledge advances through collaboration and 
the context of a social system with well developed norms.” 


As shown in Exhibit 7, 82 percent of 4-H Science programs engage youth in activities 
that encourage them to work in teams or small groups—allowing youth to develop interpersonal 
skills as they master science content—and 61 percent of programs integrate activities that require 
youth to build or construct models to demonstrate scientific phenomena. Youth also have 
opportunities to practice science-related skills, including gathering data or information (52 
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percent), developing questions and conducting research (45 percent), and analyzing and 
interpreting data (43 percent). 


Exhibit 7 
Program Activities Occurring at Every or Nearly Every Meeting, 
In Percents (n=182) 


Work together onteams orin groups 
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look for answers 


Analyze or interpret information or data 


Conduct expenments 


Present data or informationto a group 


Usetools or technology 


Exhibit reads: Eighty-two percent of programs engage youth in activities in which they work together on teams on in 
groups at every or at nearly every meeting. 


Eighty-nine percent of science educators reported that they engage youth in activities that 
allow them to learn about careers that use science, and 81 percent of science educators reported 
using activities that involve meeting with adults who work in science-related fields (Exhibit 8). 
Providing youth with opportunities to learn about careers in science is a practice in line with 
current research that suggests that exposing youth to science early on can affect their education 
and career choices later in life (Tai, Liu, Maltese, & Fan, 2006). 
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Exhibit 8 
Activity Offerings that Feature Information about 
Science-Related Careers or Educational Pathways (n=182) 


Content Area Percent of Programs 
Learn about careers that use science Fw 


Meet with adults who work in science-related fields | st 
Learn about educational choices that youth must make 
to pursue a science-related career 


Take field trips to see how science is used in real life 


Exhibit reads: Eighty-nine percent of programs engage alee = in activities 
that allow them to learn about careers that use science. 


Planning and intentionality. Eighty-one percent of educators reported using lesson 
plans to guide activities at some point; of this group, less than half reported using lesson plans 
consistently for all program meetings (42 percent). 


Program Volunteers and Staff 


Having knowledgeable, caring, and capable staff members is central to delivering high 
quality out-of-school time programming (Durlak & Weissberg, 2007; Russell, Mielke, & 
Reisner, 2009). In this section we describe the characteristics of staff members leading 4-H 
Science programming, as provided by science educator survey respondents. 


Characteristics of science educators. The data we collected on science educators 
represents both volunteers and paid staff members who lead 4-H Science programs. Many 
science educators leading programs held degrees in science-related fields and/or had previous 
experience working with youth. Eighty percent of educators reported having attained an 
undergraduate or graduate degree (Exhibit 9). Among educators who received a college or 
graduate degree, 43 percent majored in a science-related field. Thirty-five percent of 
respondents had previous volunteer or work experience in youth programs with a science focus. a 


'° Many of the staff background questions were also part of a common set of staff measures used by Noyce grantees. 
Please see Appendix B for a complete copy of the staff survey instrument, which notes which items are part of the 
Noyce staff set. 
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Exhibit 9 
Science Educator Background (n=183) 
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Exhibit reads: Ninety-seven percent of science educators who lead programs are 21 years old or older. 


Undergraduate Degree 7] 
Two-Year College Completion 


Program staffing. Aside from the primary science educator, 75 percent of all science 
programs involved additional volunteers or staff. Of these programs, 67 percent involved adult 
volunteers with science backgrounds, 65 percent involved parent volunteers, 47 percent involved 
adult volunteer without science backgrounds, 33 percent involved high school students, and 18 
percent involved college students. Programs often used a mixture of staff and volunteers with 
different backgrounds and qualifications. 


Of the science educators reporting additional personnel, 71 percent reported that their 
programs involved 10 or fewer additional volunteers or staff. Thirty-six percent had just one or two 
additional volunteers or staff. Fourteen percent of science educators reported involving between 11 
and 20 additional staff or volunteers. 


Program Supports 


Science educators rely on several supports to maintain program operation, including 
opportunities to discuss science programming with county- and state-level 4-H leaders, training 
and professional development opportunities, and financial contributions. 


Training and support from 4-H. A majority of science educators reported that they spoke 
with county and state 4-H representatives at least once per year (89 percent). Forty-three percent 
of science educators reported speaking with representatives several times per year, and 20 percent 
of science educators spoke with state and county representatives a few times per week. 
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4-H works to provide training opportunities for volunteers and staff in positive youth 
development, science content areas, and instructional methods that support program quality and 
activity delivery. When asked about the adequacy of these trainings, survey respondents were 
overwhelmingly positive: 90 percent of educators agreed that the training and support they 
received in youth development has been adequate, and 80 percent responded similarly about the 
trainings offered to support science content delivery. 


Exhibit 10 
Science Educators Who Agree That Their Training Has Been Adequate, 
In Percents (n=180) 


Thetraining and support | have 
receivedin positive youth 
development has been adequate 


The training and support | have 
receivedin science content has been 
adequate 


Thetraining and support | have 
receivedin science instructional 
methods has been adequate 


Exhibit reads: Ninety percent of respondents agreed that they have received adequate training in positive youth 
development. 


Financial and in-kind supports. Because many 4-H Science programs are led by 
volunteers and have limited, if any, operating budgets, science educators reported receiving 
various financial and in-kind contributions to support program operation. While half receive 
support from 4-H, science educators also reported supporting program operations by collecting 
dues (40 percent), using personal funds and resources (40 percent), or writing grants or holding 
fundraisers (35 percent). 


Challenges 


Some programs encountered challenges to implementing programming. A lack of funding 
to purchase science-related curricula (39 percent) and limited access to appropriate physical space 
(35 percent) were among the most frequently described challenges. Lack of volunteers with 
science skills or knowledge also created a challenge for science educators (22 percent). 
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Exhibit 11 
Major Challenges to Program Implementation (n=179) 


Lack of funding to purchase science-related 
curricula 
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Exhibit reads: Thirty-nine percent of program leaders view lack of funding to purchase science-related curricula as a 
major challenge to implementing programming. 


4-H Science Participants 


In the section that follows, we present demographic information on the youth who 
completed YEAK surveys, describe their skills and attitudes toward and skills in science, and 
compare the attitudes held by 4-H Science participants to those of a national sample of youth. 
We also present comparisons among 4-H Science participants based on gender, race/ethnicity, 
and exposure to 4-H Science programming. 


As described in the introduction, because we used a sampling procedure that drew a 
random selection of programs, surveyed all participants in the sampled programs, and achieved 
high response rates, we believe that these youth survey data are representative of 4-H Science 
participants in the seven evaluation states. 


Because the youth sampling procedure used for this year’s report was different than the 
procedure used in Year 1, and because the LGUs involved in the evaluation differed across Year 
1 and Year 2, we do not make any comparisons between Year | and Year 2 youth survey data. 
We do, however, provide a narrative description of similarities and differences across Year | and 
Year 2 findings in Appendix E. 
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Demographics 


We received surveys from a total of 486 youth ranging from 9 to 18 years of age. Sixty- 
nine percent of youth were between 9 and 12 years of age, and 31 percent were between ages 13 
and 18. Females comprised the majority of the youth sample (59 percent). Sixty-one percent of 
youth survey respondents reported their race or ethnicity as white, 25 percent as African 
American, 12 percent as other, 8 percent as Hispanic/Latino, 4 percent as Native American, 3 
percent as Asian-American, and 1 percent as Native Hawatian/Other Pacific Islander. (Youth 
could select more than one race/ethnicity.) 


More than three-quarters (81 percent) of youth reported they attend public school. Ten 
percent said they were home schooled, six percent attended a religious school, and three percent 
attended private school. 


Exposure to 4-H 


In formal school-based learning environments, rote learning often takes precedent over 
inquiry and exploration as teachers and schools react to the demands of assessments that 
prioritize memorization (President’s Council of Advisors on Science and Technology, 2010). 
Informal settings, such as after-school programs, clubs, and camps, can give youth the 
opportunity to learn science content and process skills in engaging and interactive ways. Such 
environments also have the potential to provide rich youth development experiences. 


In order to understand the experiences that surveyed youth have had in 4-H and the 
background they bring to their current 4-H Science program, we asked youth to report their 
previous involvement with 4-H and the extent of their involvement with their current program. 


Youth experience in 4-H. One of the goals of the 4-H Science Initiative is to attract new 
youth to 4-H through science programming. Many of the youth surveyed in this evaluation were 
indeed new to 4-H: 44 percent of respondents said that this was their first year. There was also a 
large group of youth with more extensive 4-H experience: 17 percent of respondents were in 
their second year, and 39 percent had been in 4-H for three or more years. Thirty-six percent of 
youth reported that they are or have been involved in another science-related 4-H program other 
than their current program. 


When asked for how long they have been participating in their current science project or 
program, 37 percent of youth said they have been in the program for one month or less. Eleven 
percent have been participating for 2 to 4 months, 11 percent for 5 to 7 months, and 41 percent 
for 8 months or more. 


Extent of weekly involvement. Participants spend varying amounts of time each week in 
their current 4-H Science programs. Roughly one-quarter of youth reported that they spend one 
hour or less each week on their project (28 percent), and a quarter spend between one and three 
hours each week (25 percent). Nearly half of respondents (47 percent) spend more than three 
hours per week in their program. These variations may be due to different levels of youth 
attendance at programs, but may also reflect the variety of ways that 4-H science programming is 
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delivered: in after-school settings, during school, in clubs during the week or on the weekends, 
or in a camp. 


Delivery mode. Because 4-H offers several different programming models in which 
youth can participate, we asked youth to indicate in which types of 4-H programs they 
participate. Youth reported participation in a range of 4-H programming, with close to half (46 
percent) involved in clubs, 42 percent in local fairs and events, and 41 percent in at-home 
projects (Exhibit 12). Over half (56 percent) of youth reported participating in more than one 
type of 4-H program. 


Exhibit 12 
Youth Involvement in 4-H Programming (n=400) 


Program Type Percent of Respondents 
Local fairslevents 


Exhibit reads: Forty-six percent of youth are involved in 4-H clubs. 
Note: Percentages do not add to 100 because youth could select more than one 
type of programming. 


Attitudes Toward Science 


One of the goals of the 4-H Science Initiative is to increase participants’ interest in 
science in school, in their personal lives, and in their communities. The youth survey included a 
series of questions intended to gauge respondents’ enthusiasm for science not only as an 
academic subject but also as a subject that touches their everyday lives. 


Noyce Enthusiasm for Science youth survey items. In order to begin to build a broad set 
of data on interest in science among youth enrolled in informal science education programs, the 
Noyce Foundation worked with Dr. Cary Sneider of Portland State University, Dr. Gil Noam of 
Harvard University, and Foundation grantees to develop a set of youth survey items, collectively 
referred to as the Noyce Enthusiasm for Science items. The Noyce items were adapted from 
various science-related youth survey instruments, and their psychometric properties were tested 
by researchers at Harvard’s Program in Education, Afterschool, and Resiliency (PEAR). We 
included these items on the 2011 version of the YEAK survey. 


Youth responses to the Noyce Enthusiasm for Science survey items suggest that youth 
who attend 4-H Science programs are highly engaged in science-related activities, and are eager 
to participate in science activities, especially in informal settings. Most participants reported that 
they enjoy visiting science museums or zoos, like to see how things are made, and like to 
participate in science projects. Youth were less likely to report that they enjoyed reading print 
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materials about science; however, this may simply reflect participants’ increasing use of 
electronic media and decreasing use of print materials (Exhibit 13). 


Exhibit 13 
Noyce Enthusiasm for Science Survey Items, 
In Percents (n=389) 
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| like to see how things are made 

| like to participate in science projects 

| wantto understand science 
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Percent of 4-H Science Youth Participants 
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Exhibit reads: Forty-five percent of youth strongly agreed and 42 percent agreed that they enjoy visiting science 
museums or zoos. 
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Overall, young people in 4-H science programs are interested in and engaged in science 
activities. Their scientific interests may have been present before they joined their 4-H science 
program—indeed, they may have joined 4-H science programs precisely because they already 
had a strong interest in science. Although we cannot isolate the impact that participation in 4-H 
science programs may have had on youths’ engagement and interest in science, these results 
show that programs are serving youth whose science interests could be intensified or sparked by 
4-H science programming. 


National Assessment of Education Progress (NAEP) science assessment comparisons. 
In order to compare 4-H participants’ attitudes toward science against those of a representative 
sample of youth, we included a set of items on the survey that were taken from the NAEP 
Science assessments from 2005 and 2009.'" '? The NAEP Science assessment is administered to 
a nationally representative sample of fourth-, eighth-, and twelfth-grade students roughly every 
four years and includes a set of items designed to measure respondents’ interest in science. 


In this section we present the results of comparisons between 4-H participants’ NAEP 
item responses and those of NAEP assessment youth. For youth in the fourth and eighth grades, 
we were able to disaggregate the NAEP data by state, allowing us to compare 4-H participants’ 
responses only to NAEP responses in their states. ° Comparisons of twelfth-grade youth are not 
conducted at the state level because the NAEP assessment does not sample twelfth-grade youth 
by state. 


In general, youth in 4-H Science programming were more enthusiastic about science than 
were their peers in the NAEP sample. While the differences between the groups are apparent, 
the data do not tell us why 4-H youth are more enthusiastic about science, only that there is an 
association between enrollment in 4-H Science and youth reports of their interest in science. 
This association may exist because 4-H Science programs attract youth who have a pre-existing 
interest in science and who therefore give more enthusiastic responses to items designed to 
measure their engagement in science. 


While most of the comparisons between 4-H Science participants and youth in the NAEP 
sample were statistically significant—and likely reflect real differences in the levels of interest in 
science between the two groups—the effect sizes were not large enough to meet this evaluation’s 
threshold because of the large difference in the weighted NAEP sample sizes and the 4-H 
Science sample size. For this reason, we suggest interpreting these results with caution, and 
using them as anecdotal evidence of the level of interest that 4-H Science participants have in 
science. 


Fourth-grade results. Compared to fourth-grade youth in the 2005 NAEP Science 
sample, fourth-grade youth enrolled in 4-H Science programs generally reported higher levels of 
engagement in science activities and greater enthusiasm for science. Fifty-five percent of fourth- 


'' The NAEP science assessment did not include all science attitudinal items in both the 2005 and 2009 assessments. 
For information on which NAEP items were asked in which administration years, please refer to Appendix D. 

'? Some of the NAEP items were also part of the Noyce Enthusiasm for Science item set; please see Appendix B for 
more information. 

'S Because Iowa did not participate in the 2005 NAEP, fourth-grade and eighth-grade 4-H science participants from 
Iowa are excluded from these analyses. 
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grade 4-H Science participants agreed that they are good at science, compared to 42 percent of 
fourth-graders in the 2005 NAEP sample (Exhibit 14). The largest difference between the two 
groups existed for the item that asked if youth believe science is useful for solving everyday 
problems. Sixty-one percent of fourth-grade 4-H Science youth agreed with the statement, 
compared to only 40 percent of fourth-grade youth in the NAEP sample. 


Exhibit 14 
Fourth-Grade 4-H Science and 2005 NAEP Respondent 
Attitudes Toward Science, in Percents 
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@ Fourth-Grade 4-H Science Participants (n=122) ™ 2005 Fourth-Grade NAEP Sample (n=22,400) 


Exhibit reads: Seventy-five percent of fourth-grade 4-H Science participants agreed that they like science, compared 
to 65 percent of 2005 fourth-grade NAEP respondents. 


*Indicates a statistically significant comparison (p < 0.05) 


Note: 4-H respondents ages 9 and 10 were classified as fourth-graders for this analysis. 
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Eighth-grade results. Similar to fourth-grade participants, eighth-grade 4-H Science 
participants were also more enthusiastic about science than were youth in the NAEP sample of 
youth (Exhibit 15). For example, 61 percent of eighth-graders in 4-H Science programming 
agreed that they like science, compared to 51 percent in the NAEP sample of eighth-grade youth. 
Only 14 percent of eighth-grade youth in 4-H Science programs agreed that science is boring, as 
compared with 30 percent of eighth-grade youth in the NAEP sample. 


Exhibit 15 
Eighth-Grade 4-H Science and 2005 NAEP Respondent 
Attitudes Toward Science, in Percents 


® 
2 
tor) 
© 
° 
c= 
= 
c= 
~ 
=] 
°o 
> 
_ 
° 
-~ 
i= 
o 
2 
® 
a 


llikescience Science is useful for lam goodatscience Science is boring* 
solving everyday 
problems* 


BEighth-Grade 4-H Science participants (n=51) 2005 Eighth-Grade NAEP Sample (n=22,700) 


Exhibit reads: Sixty-one percent of eighth-grade 4-H Science participants agreed that they like science, compared to 
51 percent of eighth-grade youth in the 2005 NAEP sample. 


“Indicates a statistically significant comparison (p < 0.05) 


Note: 4-H respondents ages 13 and 14 were classified as eighth-graders for this analysis. 
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A greater number of eighth-grade youth agreed that science is one of their favorite 
subjects than did youth in the NAEP sample (64 percent compared to 46 percent), as shown in 
Exhibit 16. However, roughly equal percentages of eighth graders in both the YEAK and NAEP 
samples reported that they take science only because it will help them in the future (60 percent 
compared to 56 percent). This suggests that while 4-H Science participants enjoy science more 
than their peers, they might not participate in science activities if they did not perceive the 
potential for future benefit. 


Exhibit 16 
Eighth-Grade 4-H Science and 2009 NAEP Respondent 
Attitudes Toward Science, in Percents 
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@BEighth-Grade 4-H Science Participants (n=53) | ™2009 Eighth-Grade NAEP Sample (n=19,900) 


Exhibit reads: Thirty-eight percent of eighth-grade 4-H Science participants agreed that they take science only 
because they have to, compared to 48 percent of 2009 eighth-grade NAEP respondents. 


*Indicates a statistically significant comparison (p < 0.05) 


Note: 4-H respondents ages 13 and 14 were classified as eighth graders for this analysis. 
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Twelfth-grade results. Like their fourth- and eighth-grade counterparts, twelfth-grade 
youth participating in 4-H Science programming reported higher levels of engagement in 
science. These differences are more pronounced at this age group, suggesting that youth 
participating in 4-H Science activities at this age level have a strong, established interest in 
science. 


As shown in Exhibit 17, 67 percent of twelfth-grade 4-H Science participants agreed that 
they would like to have a science-related job, compared to 37 percent of twelfth-grade youth in 
the national sample. Similar percentages of twelfth-graders in each group agreed that they only 
take science because it will help them in the future (51 percent in both groups). Again, this 
suggests that while 4-H youth are more engaged in science than are their peers, they might not 
enroll in science activities if they did not foresee future benefits. While this is not necessarily a 
finding that should be interpreted negatively, it is interesting to note how youth perceive the 
potential utility of science in their future education or career. 


Exhibit 17 
Twelfth-Grade 4-H Science and 2009 NAEP Respondent 
Attitudes Toward Science, in Percents 


® 
® 
—_ 
D 
© 
° 
£ 
= 
1 
= 
3 
° 
~ 
- 
° 
a4 
ro 
® 
rs) 
— 
o 
o 


When | graduate from high Itake science only Itake science only Science is one of my 
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Exhibit reads: Sixty-seven percent of twelfth-grade 4-H Science participants agreed that they would like to have a 
science-related career after graduating from high school, compared to 37 percent of 2009 twelfth-grade NAEP 
respondents. 


*Indicates a statistically significant comparison (p < 0.05) 
Note: Sampling for the twelfth-grade NAEP assessment is not conducted by state, and no state-level data are 


available. Data depicted here reflect the 2009 national NAEP sample of twelfth-grade youth. 4-H respondents ages 
17 and 18 were classified as twelfth-graders for this analysis. 
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Educational and Career Aspirations 


The 4-H Science Initiative seeks to expose youth to career opportunities in science fields. 
This goal is in line with current national goals related to science: the National Research 
Council’s Committee on Highly Successful Schools or Programs for K-12 STEM Education, in 
setting out goals for STEM education in the United States, highlighted the importance of 
preparing people for science-related careers (NRC, 2011). 


4-H science programs are well-positioned to influence young people’s career aspirations: 
research has shown that young people’s interest in science careers may solidify at an early age. An 
analysis of longitudinal data from youth participating in the National Educational Longitudinal 
Study of 1988 points to the importance of students’ interest in science careers as early as eighth 
grade in their eventual pursuit of a science major (Tai et al, 2006). Participating in a 4-H Science 
program at a young age could lay the groundwork for pursuing a science career later in life. 


Youth in 4-H Science programming reported having high educational aspirations. When 
asked how far they want to go in school, half of youth surveyed (52 percent) want to finish 
college, while an additional 37 percent reported that they want to get more education after 
completing a college degree. Three percent said they would like to go to college for a little 
while, 2 percent said they wanted to attend a vocational school, and 7 percent said they wanted to 
graduate from high school. When we asked youth if they wanted to pursue a science-related 
career after graduating from high school, 54 percent of youth agreed or strongly agreed. 


Participation in Community Science Activities 


The 4-H Science logic model lists helping youth apply their science-related skills to solve 
everyday problems in their communities as a short-term outcome that 4-H Science programs 
should endeavor to achieve with youth. This goal of developing citizen-scientists reflects aspects 
of our recent national conversations about the importance of science education. Among 
researchers, educators, and policymakers, there is increasing agreement that science education in 
both formal and informal contexts should focus on helping young people understand how science 
functions in their everyday lives. Because active engagement in public issues increasingly relies 
on citizens’ familiarity with scientific issues, there is increasing agreement that informal and 
formal education programs should help youth develop this knowledge (Bevan et al, 2010; and 
National Academies of Sciences Committee on Science Learning K-8, 2007). 


As shown in Exhibit 18, the majority (70 percent) of 4-H Science participants reported 
participating in a science-related community service project in the past year, while a little more 
than half of youth (56 percent) have taught others about science. One-third of youth organized 
science-related events (35 percent) or used science tools to help their community (33 percent). 
(Items adapted from Silliman, 2010.) 
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Exhibit 18 
Participation in Community Science Activities (n=4317) 


In the past year, | Percent of 
area salah Respondents 


Exhibit reads: Seventy percent of youth in 4-H Science programs reported that they 
have helped with a community service project related to science in the past year. 


Life Skills 


The 4-H Science logic model identifies the acquisition and use of life skills as intended 
outcomes for participants. 4-H Science programs are expected to provide learning opportunities 
that support the development of life skills such as decision making, critical thinking, and 
problem solving. We asked youth to provide a self-report on their own skills in each of those 
areas, using items adapted from Perkins and Mincemoyer (2002). 


Decision making. Nearly half of youth surveyed reported that they always think before 
making a choice, while 40 percent said that they think about all the information they have about 
different choices before coming to a decision. Thirty-nine percent reported that they always look 
for information to help them understand the problem at hand, 36 percent always think of past 
choices when making new decisions, and 30 percent consider the risks of a choice before making 
a decision (Exhibit 19). 
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Exhibit 19 
Decision Making Skills, in Percents (n=457) 


When | have a decision to make... 


| think before making achoice 


| think about all the information | have about the different 
choices 


| look for information to help me understand the problem 


| think of past choices when making new decisions 


| considerthe risks of achoice before making a decision 


20 40 60 80 
Percent of Respondents 


@ Always B Usually 


Exhibit reads: Forty-six percent of 4-H Science participants reported that they always think before making a choice; 
28 percent of participants said that they usually do this. 


Critical thinking. One-third (35 percent) of youth in 4-H Science programming said that 
they always keep their mind open to different ideas when planning to make a decision. Similar 
numbers of youth reported that they always: compare ideas when thinking about a topic (31 
percent), easily express their thoughts on a problem (30 percent), have more than one source of 
information before making a decision (28 percent), and are able to tell the best way of handling a 
problem (27 percent), as seen in Exhibit 20. 
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Exhibit 20 
Critical Thinking Skills (n=447) 


When | think about things... 


| keep my mind open to different ideas when 
planning to make a decision 


| compare ideas when thinking about a topic 


Ican easily express my thoughts on a problem 


lusually have more than one source of information 
before making adecision 


lam able to tell the best way of handling a problem 


60 
Percent of Respondents 


m@ Always B Usually 


Exhibit reads: Thirty-five percent of 4-H Science participants reported that they always keep their mind open to 
different ideas when planning to make a decision; an additional 39 percent of participants reported that they usually 
do this. 


Problem solving. As displayed in Exhibit 21, half of youth respondents (49 percent) said 
that when solving a problem, they always first figure out what the problem is. Thirty-two 
percent of youth reported that they always think about how their solution worked after they have 
solved a problem. Similar numbers of youth said that they always: try to determine what caused 
a problem (31 percent), compare each possible solution with others to find the best one (26 
percent), think before putting a solution into action (26 percent), and do what they have done in 
the past to solve a problem (25 percent). 
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Exhibit 21 
Problem Solving Skills, in Percents (n=445) 


When solving aproblem... 


I first figure out exactly what the problemis 


Once | have solved a problem, | think about howmy 
solution worked 


I try to determine what caused it 


| compare each possible solution with others to find 
the bestone 


After selecting a solution, | think about it for a while 
before putting itinto action 


Ido what| have donein the pastto solve it 


20 40 60 80 
Percent of Respondents 


@ Always B Usually 


Exhibit reads: Forty-nine percent of 4-H Science participants said that when solving a problem, they always first figure 
out what the problem is. An additional 30 percent of participants said they usually do this. 


Science Process Skills 


Two of the goals of the 4-H Science logic model are for youth to use what they learn in 
4-H Science programs in other contexts, and to identify new areas where they can apply their 
science-related skills to solve everyday problems. 4-H’s goal of developing informed citizens 
with an understanding of how science works reflects many of our national conversations about 
the importance of STEM education. The possession of these science-related skills and the ability 
to understand scientific concepts are essential for being an informed citizen (National Research 
Council, 2011). 


We asked youth to rate their own abilities to perform a series of science process skills 
using a set of items adapted from Arnold and Bordeau, 2009. The items included two separate 
sets of questions, one for youth ages 9-12 and one for youth ages 13-18. 


The vast majority (87 percent) of youth ages 9-12 reported that they can make a chart or 


picture to show information, while 86 percent said that they can do an experiment to answer a 
question (Exhibit 22). 
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Exhibit 22 
Science Process Skills Inventory, Ages 9-12 (n=270) 


Percent of Respondents, 

Ages 9-12 
Make a chart or picture to show information | rsd 
Do an experiment to answer a question 


Tell others how to do an experiment 


Explain why things happen in an experiment 


87 
Write down information correctly 


Exhibit reads: Eighty-seven percent of 4-H Science participants, ages 9-12, said that 
they can make a chart or picture to show information. 


We asked youth participants ages 13 to 18 a similar set of questions about their mastery 
of certain science process skills. Most youth reported that they can always or usually: use the 
results of their investigation to answer the questions they asked (83 percent), ask a question that 
can be answered by collecting data (80 percent), or record data accurately (77 percent). 


Exhibit 23 
Science Process Skills Inventory, Ages 13-18 (n=132) 


[and Pant enon a 5 | 
[Aska question tat can be answeredby colectng data | SSR 
[Use scence testo sharemy resus ————SSCSCSC~S~S~SSSSR 
[Use scenitcknowedge tofom aquesion ———~SCS~S~—CSC“~“~*~*~a OCS 


Exhibit reads: Eighty-three percent of 4-H Science participants, ages 13-18, said that they can always or usually use 
the results of their investigation to answer the questions they asked. 


Science Program Environment and Benefits 


Informal learning programs, like those offered as a part of 4-H Science programs, have 
the potential to provide engaging opportunities to learn new skills in a context focused on 
providing positive youth development. Current research suggests that when certain best 
practices are implemented, informal and out-of-school time learning programs can have a 
positive effect on youth. These practices are associated with increased program participation, 
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greater interest in the program, and increased social and emotional skills, which can result in 
increased academic performance. An analysis of out-of-school time research identified best 
practices for programs, including the development of nurturing youth-adult relationships, 
providing diverse and engaging activities for youth, and emphasizing multiple aspects of youth 
development (Moore, Bronte-Tinkew, & Collins, 2010). 


In order to evaluate participants’ opinions of their 4-H Science program environments, we 
asked them to select the three things they liked best about their science program. Youth were most 
enthusiastic about the relationships they have at their 4-H Science programs: 65 percent of youth 
said that the opportunity to spend time with their friends was their favorite part of their program, 
while 35 percent said that the fact that adults were caring and kind was their favorite aspect. 


In addition to the relationships that youth develop in their programs, youth were 
enthusiastic about the science learning opportunities that their programs provide: 52 percent of 
youth said that getting to do hands-on science activities and projects is one of their favorite 
things about their program. 


These results demonstrate that youth are highly interested in both the social and hands-on 
nature of their science program, and also reflect practices that the research community believes 
may be critical to achieving positive outcomes with youth. 


Exhibit 24 
Favorite Characteristics of This Science Program (n=412) 


Percent of Respondents 


| get to spend time with my friends a a 
| get to do hands-on science activities and projects a 
The adults are caring and kind — 
| can use tools and materials here that | don’t have at school 

or at home 

It is a group where | feel like | belong = 


| get opportunities to demonstrate things | have learned or 

made in front of others 

It is a place where | feel safe —=. 4 
| like the curriculum/project book 
| get positive feedback from the adults and other kids 
| get to do community service a cen 


Exhibit reads: Sixty-five percent of 4-H Science participants said that getting to spend time with 
their friends is one of the things they like most about participating in their program. 


Note: Percentages do not add to 100 because participants could select up to three responses. 
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Associations between Youth Characteristics and Youth 
Survey Responses 


In this section we describe the results of analyses designed to reveal any differences 
among 4-H Science participants’ survey responses based on participant characteristics including 
gender, membership in a racial or ethnic group that is underrepresented in the science fields, and 
the extent of the respondent’s participation in 4-H Science programming. 


Differences by Demographic Characteristics 


Because 4-H is concerned with increasing the number of women and underrepresented 
minorities in the science fields, we examined whether youth survey responses differed by gender 
or by their membership in a racial or ethnic group that is underrepresented in science fields. By 
examining whether respondents gave different answers to survey items based on these 
characteristics, 4-H can consider whether their programming is effectively reaching all of the 
youth they hope to serve. All comparisons presented here meet this evaluation’s threshold for 
statistical significance and effect size. 


Gender. Boys and girls in 4-H Science programming responded differently on several 
scale items included in the YEAK survey. Overall, boys were more positive than were girls on a 
scale measuring their enthusiasm about science. However, girls gave more positive evaluations 
of their own decision making and critical thinking skills than did boys. Girls also gave higher 
average ratings to their 4-H program’s overall climate.'* 


Membership in a racial or ethnic group that is underrepresented in scientific fields. In 
an effort to examine the differences between youth from racial and ethnic groups that are well- 
represented in the science fields and those that are underrepresented, youth were categorized into 
one of two groups according to their self-reported race or ethnicity. We categorized youth who 
reported being white and/or Asian American as well-represented in science fields. Youth from 
other racial/ethnic backgrounds (Hispanic/Latino, African American, Native American, Native 
Hawaiian/Other Pacific Islander, and youth who specified “other”) were categorized as 
underrepresented. '° Using this definition, 56 percent of youth survey respondents were from 
groups well-represented in science fields, with the remaining 43 percent coming from 
underrepresented groups. 


On average, youth from underrepresented racial or ethnic groups rated their decision 
making, critical thinking, and problem solving skills lower than did youth from well-represented 
groups. Underrepresented youth also reported lower levels of enthusiasm about science than did 
well-represented youth. In addition, under-represented youth ages 13 to 18 also rated their 
science process skills lower than did well-represented youth. 


' Scale properties are detailed in Appendix C. 
'S Youth who identified as being more than one ethnicity were considered part of the underrepresented group, unless 
the only two ethnicities they selected were Asian-American and White. 
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Underrepresented youth had lower average scores on a scale measuring benefits that they 
experienced from participating in their 4-H program, and also gave lower ratings of their 
program’s overall climate than did well-represented youth. 


Finally, youth from racial or ethnic groups that are underrepresented in scientific fields 
were significantly less likely to report wanting to pursue a science-related career than did youth 
from well-represented groups. Twenty-two percent of youth from underrepresented groups 
reported wanting to pursue a science career, compared with 36 percent of youth from well- 
represented groups. 


Exposure to 4-H Science programming. We grouped youth into high, moderate, and 
low exposure categories based on their responses to two survey items about their attendance and 
persistence in their 4-H Science program: 


rT] In general, how many hours do you spend in this program/project each week? 
7 How long have you been participating in this science, engineering, or technology 
program/project? 


4-H does not currently monitor youth attendance or persistence in a program, so we 
classified youth according to their self-reported participation in 4-H Science programming. We 
assigned respondents to one of the three exposure groups: low, moderate, or high. 


Exhibit 25 
Exposure Index 


How many hours do you spend in this program/project each week? 
How long have you been 


participating in this 
program/project? 


One hour or less Between 1 and 3 hours More than 3 hours 


Low Exposure Low Exposure Low Exposure 


Yutadour-migeiie Low Exposure Moderate Exposure Moderate Exposure 
(n=35) (n=23) (n=15) 
: Moderate Exposure High exposure High exposure 
Five to seven months (n=34) (n=24) (n=11) 
: Moderate Exposure High exposure High exposure 
Eight months or more (n=41) (n=49) (n=85) 


Exhibit reads: Youth who reported participating for one hour or less each week and who have been participating in 
their project for one month or less were classified as having a low exposure to 4-H Science programming. 


Thirty-six percent of respondents were classified as low-exposure, 26 percent as 
moderate-exposure, and 39 percent as high-exposure. For purposes of significance testing, we 
only compared youth in the high- and low-exposure groups. 
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Youth in the high-exposure group gave higher ratings to their own decision making, 
critical thinking, and problem solving skills. They also reported experiencing greater benefits 
from their science program than did youth with less exposure to their science program. 


These results should be interpreted with care because youth who spend more time in their 
4-H Science program may be more enthusiastic about science than youth who spend less time 
there, and may therefore be more likely to give positive responses about their science-related 
abilities and about their science program. These youth may have entered their 4-H Science 
program with established skills and interests in science; we cannot isolate the effects of the 4-H 
Science programs themselves. 


Summary of Findings 


7 4-H Science programming represents a wide range of 4-H delivery modes and 
reaches youth ranging from 5 to 18 years old. Program enrollments reflect the 
variety of program sizes and types, with some small clubs serving only a handful 
of youth, and larger state- and county-wide programs reaching as many as 3,000 
youth. 


7 Although newer programming areas such as robotics and wind power are 
commonly associated with the 4-H Science Initiative, in the states evaluated, large 
or small animal science make up the largest single content-area block of 4-H 
Science programs (36 percent), and environmental science programs make up 23 
percent. A large majority of programs also use a curriculum developed by 4-H 
(77 percent) to guide their science programs. 


= 4-H Science educators’ stated program objectives are largely aligned with the 4-H 
Science Checklist: educators prioritize both science content and process skills 
development, as well as providing a positive youth development context for 
learning. 


7 The overwhelming majority of paid staff and volunteers who lead science 
programs are female (84 percent). More than three-quarters have a college or 
graduate degree, and just fewer than half of educators with postsecondary degrees 
majored in a science-related field. 


7 Science educators outlined several challenges to implementing their programs, 
including a lack of funding to purchase curricula, and difficulty in finding an 
appropriate space in which to hold program meetings. 


7 Among youth who responded to the YEAK survey, almost half said that they 


were in their first year of 4-H, suggesting that many youth are coming to 4-H 
through 4-H Science program offerings. 
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7 Youth are highly enthusiastic about science and they seek out opportunities to 
engage in both formal and informal science activities. Many youth aspire to enter 
scientific careers and give their own science abilities and skills high marks. When 
compared with their peers on survey items taken from the NAEP Science 
assessment, 4-H youth were far more enthusiastic about science and more 
confident in their science-related skills. 


7 Youth reported that they especially enjoyed the opportunities programs gave them 
to interact with friends and caring adults, and to participate in hands-on activities. 


7 Youth who reported attending their 4-H Science program more often and for more 
hours each week were more likely to rate their own life skills highly, to 
participate in more community science activities, and to have higher overall 
opinions of their science programs. 


7 Analyses of youth survey responses revealed that boys were slightly more 
enthusiastic about participating in science activities than were girls. However, 
girls were more likely to rate their program climates highly, and to give better 
evaluations of their own life skills. 


7 Youth belonging to racial and ethnic groups historically underrepresented in the 
science fields were less likely than youth from well-represented groups to report 
an interest in pursuing a science career, and less likely to rate their own science 
and life skills highly. Youth from underrepresented groups, on average, rated 
their program’s climate and the benefits they received from attending lower than 
did youth from well-represented groups. 


Recommendations 


7 Because some science educators reported facing challenges related to the 
affordability of science curricula, 4-H should continue to facilitate the exchange 
of curricular resources among states and counties, perhaps by continuing to 
promote online networks where educators can identify affordable curricula. 


7 Educators also noted that finding adequate space in which to host their 4-H 
Science programs was a challenge. 4-H may therefore want to collect information 
regarding best practices and field-tested tips to help educators identify and secure 
space for their programs, and then share this information with the state- and 
county-level staff who can advise their educators. 


7 In order to support and develop girls and youth from racial and ethnic groups that 
are underrepresented in the science fields, 4-H should continue to seek out best 
practices for recruiting, engaging, and supporting these youth, and to share this 
information among state- and county-level leadership. 
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Appendix A 
4-H Science Checklist 


18 USC Te? 


4-H Science Checklist 


A “Science Ready” 4-H experience is a program that is framed in Science concepts, based on Science 
standards and intentionally targets the development of science abilities and the outcome articulated by 
the 4-H Science Logic Model. Additionally, it integrates the Essential Elements and engages participants 
in experiential and inquiry based learning. In addition to the following criteria below, it’s also 
recommended that science programs offer a sustained learning experience which offers youth the 
opportunity to be engaged in programs with relevant frequency and duration. Utilize the following 
checklist to self assess the program you deliver. 


To meet the needs of children, youth and the 
nation with high-quality science, engineering and technology programs... 


National Science Education Standards - Science education standards are criteria to 
judge quality: the quality of what young people know and are able to do; the quality of the 
science programs that provide the opportunity for children and youth to learn science; the 
quality of science teaching; the quality of the system that supports science leaders and 
programs; and the quality of assessment practices and policies. 
http://www.nap.edu/readingroom/books/nses/ 


F Are you providing science, engineering and technology programs based on 


Are you providing children and youth opportunities to improve their Science 
Abilities? 


Predict, Hypothesize, Evaluate, State a Problem, Research Problem, Test, Problem Solve 
Design Solutions, Measure, Collect Data, Draw/Design, Build/Construct, Use Tools, Observe, 
Communicate, Organize, Infer, Question, Plan Investigation, Summarize/Relate, 
Invent/Implement Solutions, Interpret/Analyze/Reason, Categorize/Order/Classify, 
Model/Graph/Use Numbers, Troubleshoot, Redesign, Optimize, Collaborate, Compare 


Are you providing opportunities for youth to experience and improve in the 
Essential Elements of Positive Youth Development? 


Do youth get a chance at mastery — addressing and overcoming life challenges in your 
programs? 

Do youth cultivate independence and have an opportunity to see oneself as an active 
participant in the future? 

Do youth develop a sense of belonging within a positive group? 

Do youth learn to share a spirit of generosity toward others? 


A-1 


Are learning experiences led by trained, caring adult staff and volunteers acting 
as mentors, coaches, facilitators and co-learners who operate from a 
perspective that youth are partners and resources in their own development? 


Are activities led with an experiential approach to learning? 


Are activities using inquiry to foster the natural creativity and curiosity of 
youth? 


Does your program target one or more of the outcomes on the 4-H Science 
Logic Model and have you considered the frequency and duration necessary 
for youth to accomplish those outcomes? 
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Appendix B 
Youth and Program Educator Survey Instruments 


NATIONAL 4-H SCIENCE EVALUATION 
Survey of 4-H Science Leaders 


Dear 4-H Science Program Leaders and Facilitators: 


Thank you for taking the time to complete this survey. As part of a three-year evaluation of 4-H Science 
programming, we are surveying leaders of 4-H Science clubs and programs. This survey is designed to 
capture information about your program and your responses will help 4-H understand how 4-H Science 
programs are being implemented, nationwide. Your responses are confidential; no individuals or 


programs will be identified in evaluation reports. 
If you have any questions, please contact Joy Sanzone at www.policystudies.com. 


Thank you for your help! 


Program Structure 


1. Do you lead a 4-H Science program? For example, a program that focuses on science, 
engineering, technology, or applied math? 


Select one 
Yes L] 
No [Respondent will exit the survey] L] 


2. Please provide the name of the 4-H Science program that you lead in the space below. 


3. Will this program be running in March, April, or May of this year? 
Select all that apply 


March 

April 

May 

June 

None of the above 


OO00U 


4. When does this program typically take place? 


Select all that apply 


During the school day 

After school 

On the weekend 

Over the summer 

Other (Please specify: ) 


OOU0U 
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5. Please indicate how often youth in the program meet as a group with you or other 


staff/volunteers? . 


Enter a number 


On how many days per month does the program meet? 


For approximately how many hours per month does the program 
meet? 


6. We’d like to know how long this program’s current programming cycle will be running. Please 


indicate the first day that this program met and the last scheduled meeting date. 


Enter a date 


Program period start-date 


Program period end-date 


Not applicable, program runs continuously or year-round 


O 


Use this space to briefly describe why you are unable to provide a start- and/or end-date: 


7. We would like to get a sense of how many youth are enrolled in this program and how 


frequently they attend. 


Enter a number 


Please indicate how many youth are enrolled in the program 


On average, how many youth attend each program meeting? 


8. What ages (in years) are program participants? 
Select all that apply 


18 
Other (Please specify: _—) 


@ 
OOOOOUOUOOOOOUO 0 
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9. How do most youth learn about and enroll in this program? 
Select all that apply 


Flyers distributed around the community 
Word-of-mouth 

4-H website 

Through other 4-H activities 

School 

Other (Please specify: ) 


OUOO0U00 


Program Content 


10. What are the main content areas that this program addresses? (Select up to two content 
areas.) 


Select one 


Animal Science-Large Animal 
Animal Science - Small/Companion Animal 
Veterinary Science 

Food Science 

Consumer and Family Sciences 
Environmental Science 
Environmental Stewardship 

Earth Science 

Weather and Climate 

Physical Sciences 

Plant Science 

Gardening 

Horticulture 

Technology 

Engineering 

Aerospace/Rocketry 

Computer Technology 

Geospatial Technology (GIS/GPS) 
Robotics 

Other, please specify 


OOOOUOOOOOOUOUOOOOoOoo 


11. Does this program use any of the following 4-H Science curricula to guide activities? 
Select one 


O 


Animal science curricula (poultry, beef, etc.) 
The Power of Wind 

There’s No New Water 

Robotics 

Exploring Your Environment 

Acres of Adventure 

No, we do not use a 4-H Science curriculum 


Other (Please specify. is) 


OOUU0U0O 
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12. Does this program use a published curriculum created by an organization other than 4-H in 


order to guide activities? 


Select one 
No L] 
Yes (Please specify: ) LJ 


13. Do you create, or do you require other staff or volunteers to create, lesson plans for each 


program meeting? 


Select one 
No, we do not use lesson plans L] 
We use lesson plans once in a while, but not regularly L] 
We usually use lesson plans, but not always LJ 
Yes, there are lesson plans for each program meeting L] 
Other (Please specify: ) LJ 


14. To what extent is each of the following an objective of your 4-H Science program? 


Program objectives 


Select one 


Major 
Objective 


Minor 


Objective 


Not an 
Objective 


Help youth develop pro-social and interpersonal skills 


O 


O 


O 


Provide hands-on, experiential learning activities 


Use national science education standards to inform and guide 
activities 


Teach youth about a specific science content area (e.g., 
computer programming, animal behavior, plant science, etc.) 


Help youth practice science-related skills (e.g., pose questions or 
hypotheses, collect data, (de)construct an object, analyze 
information, present findings or results, etc.) 


Help youth become aware of the roles science plays in their 
everyday lives 


Encourage youth to develop an interest in science 


Help youth understand how science can help them to improve 
their communities 


Other (Please specify) 


ab el FeV a NT 


an FV OV a 


oD 


Use this space to briefly describe “Other” objectives: 


15. Of the items you identified as major objectives, how easy or difficult is each one to achieve. 


Program objectives 


Select one 


Very 
Easy 


Easy | Difficult 


Very 
Difficult 


Help youth develop pro-social and interpersonal skills 


Provide hands-on, experiential learning activities 


Use national science education standards to inform and guide 
activities 


Teach youth about a specific science content area (e.g., 
computer programming, animal behavior, plant science, etc.) 


Help youth practice science-related skills (e.g., pose 
questions or hypotheses, collect data, (de)construct an 
object, analyze information, present findings or results, etc.) 


Help youth become aware of the roles science plays in their 
everyday lives 


Encourage youth to develop an interest in science 


Help youth understand how science can help them to improve 
their communities 


Other (Please specify) 


OHO OO) OU ;o;oodo 


EEN ed EC) ad) Ted es eal] 


Oi pa a | 


Ne CE aC) SIE) Ea ESE] 


Please discuss why each of these items is easy or difficult. 


16. To what extent is each of the following a challenge to implementing your 4-H Science 


program? 


Challenges 


Select one 


Major 
Challenge 


Minor 
Challenge 


Nota 
Challenge 


Low enrollment 


Irregular youth attendance 


Limited access to appropriate physical space 


Limited access to necessary supplies (e.g., GPS devices, 
computers, etc.) 


Lack of volunteers with science skills or knowledge 


Lack of funding to purchase science-related curricula 


Lack of science-related training opportunities for 
volunteers/staff 


Other (Please specify) 


OO OO] 0 OO) 


OO OO} 0 OF) O 


es ae | | 


Use this space to briefly describe “Other” challenges: 
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17. Please indicate how often, if ever, youth engage in the following activities as part of this 
program. 


Select one 
At Nearly : 
Activities freee ry Every ae i. Never 
9 Meeting 

Take field trips to see how science is used in real life L] L] [=I] L] 
Meet with adults who work in science-related fields i L] L] L] 
Learn about careers that use science L] L] ft] L] 
Learn about the educational choices that youth must 

make to pursue a science-related career U U U U 
Work together on teams or in groups L] L] EI al 
Build or construct things L] L] L] L] 
Develop questions and then research or look for answers LJ [@] [i] L] 
Design a test to explore an idea or hypothesis L] LJ L] L] 
Gather data or information = [] L] LJ 
Use tools or technology (e.g., GIS/GPS, computer, scale) (io) [3] (] L] 
Conduct experiments L] LJ L] LJ 
Analyze or interpret information or data ia im L] fe] 
Present data or information to a group L] [| a L] 
Design a test to explore an idea or hypothesis L] [az] L] L] 
Use computer software to design or build something 

(e.g., website, programming for a robot) O U O O 
Use the internet for research OC LJ eal LJ 
Other (Please specify) im LJ L] LJ 


Use this space to briefly describe “Other” activities: 


Program Supports 


18. On average, how often do you discuss issues related to providing science programming with 4- 
H representatives from your county or state? 


Select one 
About once a week L] 
A few times a month L] 
Several times a year L] 
Once a year L] 
Never L] 
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19. How much do you agree or disagree with the following statements? 


Select one 
Strongly Agree Disagree Strongly 
Agree Somewhat Somewhat Disagree 

The training and support | have received in 
positive youth development has been adequate O O U O 
The training and support | have received in 
science content has been adequate U U U U 
The training and support | have received in 
science instructional methods has been L] Eg] L] L] 
adequate 
| receive adequate support from my county 4-H 
office U U U U 
| receive adequate support from my state 4-H 
office U U U U 


20. What type of financial and in-kind supports currently help keep this program running? 


Select all that apply 


Fees or dues from participants 


In-kind or financial contributions from a business or organization outside 


of 4-H 


Local 4-Hoffice 


State 4-H office 


| contribute money or donate supplies 


Other (Please specify: ) 


OOOO UO oO 


Program Volunteers/Staff 


21. In addition to you, do any other volunteers or staff members help lead this program ona 


regular basis? 
Select one 
Yes 3] 
No (skip to question 23) L] 


22. How many staff and/or volunteers in your 4-H Science program fit in each of the categories 
below? (Write a number on each line. If no staff fit into a category, write “0.” If staff/volunteers 
fit in multiple categories, pick the one that you believe best describes them.) 


Enter a number 


Parents of participants 


Other adults with science-related expertise 


Other adults without science-related expertise 


College students who are science majors 


College students who are education majors 


Other college students 


High school students 


TT] 
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Noyce Staff Items 
23. Are you: 


Select one 
20 years old or younger L] 
21 years old or older L] 
24. Are you: 
Select one 
Male L] 
Female [3] 
25. What is your highest level of education? 
Select one 
Middle school (skip to question 27) [| 
High school diploma or GED (skip to question 27) al 
Some college coursework L] 
2-year college completion L] 
Undergraduate degree L] 
Graduate degree L] 
26. If you attended college, did you major in science, technology, engineering or mathematics? 
Select one 
Yes ES] 
No L] 
27. Are you currently in school? 
Select one 
No L] 
Yes, in high school L] 
Yes, in college or university L] 
Yes, other [is] 


28. For how long have you worked or volunteered in a youth development program? 


years 


29. Before leading this program, did you previously work or volunteer in a youth program with a 


science, technology, engineering, or mathematics focus? 


Select one 
Yes LJ 
No LJ 
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30. In what industry do you work? 


Select one 


Farming or ranching 


Military, police, or security 


Banking, finance, or accounting 


Computers and technology 


Research 


Medicine (doctor, nurse, lab technician) 


Teacher/Education, please specify 


Arts (writing, dancing, painting) 


Skilled trades (carpenter, plumber) 


Retail 


Engineering or architecture 


Law 


OUOUOUOOUOOUOO 


Other, please specify 


[If respondent indicated that their program will be in session in March, April, May, or June at the beginning 
of the survey, ask them to complete this contact info question.] 


A random sample of 4-H Science programs will be selected to administer Youth Engagement, 
Attitude, and Knowledge (YEAK) surveys to youth participants this spring. Please provide a 
mailing address and phone number that we can use to send YEAK surveys to you this spring in 
the spaces below. If your program is selected as part of the random sample, we will contact you 
and let you know. 


Name: 
Address: 
City: 
State: 

Zip: 

Phone number: 
Email address: 


THANK YOU! 
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4-H Science Youth Survey 


Dear Participant: 


You are being given this survey because you are part of a 4-H program or project that has to do with 
science, and we are surveying young people like you to learn about your experiences. 


This survey is voluntary. If you do not want to fill out the survey, you do not need to. However, we 
hope you will take a few minutes to fill it out because your answers are important. 


This survey is private. No one at your school, home, or 4-H program or project will see your answers. 
Please answer all of the questions as honestly as you can. If you are uncomfortable answering a 
question, you may leave it blank. 


This is not a test. There are no right or wrong answers, and your answers will not affect your 
participation or place in the program in any way. 


Thank you for your help! 


1. How many years have you been participating in 4-H? (Select ONE.) 


a. LAISIS MY TSE YOON e.2tietie Sai Aa nite iietiees Asie Malan Gane 1 
b. This 1S MY SCCON YOAN o.oo... ceeeceeeeeecceeeeeeececeeeeeceeeeeceeeeneceeeeaeaeeesneaeeesneeeecesneeeeeessieeeeeesinaes 2 
Cc. Uhr66"OF MOL: VOarS...cc ski aiaitien iseettien abies tladebeedtealagsaieetibes 3 


2. How long have you been participating in this science program/project? 


(Select ONE.) 
a. AMOMUNON TOSS sesesicpeecee ses vaaddhe Fedde svaesahecb dad sade Se eae adc ts sad eee eee 1 
b. ‘FWo:to four Months: 224s herve fib dogecess oh agen cdetes ae das Acaae nes ahdadednacdeeteas Jute leetes dese 2 
C FiVE:1O, SEVEN: MONINS treccei eiiestecbesereceetanseototacs ec ceed eendebieas Hvvcanseeceietubscbiiesus chine eanetaeeeatehesees cad 3 
d. Eight: Months OMMOre svcccccdteevecteretesctateeivleteer etn eveetatabenntate eevtguteevendiewe deleteaeteteenvdeiteaeetcad 4 


3. Have you ever been in another 4-H program or project about science? (Select ONE.) 


a. B= Ee eT 1 
b. INO ieee eh oe eS a es Se en ae 2 
Cc. DONC KMOW 82 saceiees Steele dive blo eees eek ov ee edd AL oe ad Ey 3 


4. In general, how many hours do you spend in this program/project each week? (Select ONE.) 


a. One hour or less Gach Wee K o.oo... cece cece cc cccc cece ee cececececesueeeeeceessuaeaueesesesseuseaueaseseeeeueeananeesees 1 
b. Between one and three hours Gach Wee K 00... cece eee cece cece eeccsseeeeeeseeeueseeeeesessuueeeeeuaueeseeeaseees 2 
Cc. More than three hours Cach WeEeK .......ccccccccccccceeecccececesseeeeceeseeeeeeeeaeuueeeueuseesuuaaeeeuauaeesenaaeeea 3 


5. What types of 4-H programs or projects are you involved in? (Select ALL that apply.) 


a. GIUDS es be steeleeveteeaventgbeaictes ig ven tledaved lea vate Sevean cts ael bi nar sees iva sheeelias eee teanedevtevepeetenedtead 1 
b CAMPS i iscsecheciesee tert eheeeedaereslehecacte cls petete leaned are duc beedesaevs butch eae cevaresiucles beeeels rete eaeatns 1 
c After-School programy :.vsscsicsscteeietieatevevecebeiticeedtystecheedeenetaveligerehenietiesdigheebneearnageeitiay 1 
d. LOCal TaIS/OVE MS seizes. coed esceeebecbecdzdeevbebetetebeened. eeu eecleadveee ee be beet giv etavede beclecdateevbet teeeiaanetayy 1 
e Community Service: projects: -.:.::siceeats eeagedearecsutectectecdeeests re ticerelneceteecieieeaeteesa ben etenetcas 1 
f. Working on my projects at NOME ......... eee ceeeeeecceeeeeeeceeeeeeeceeeeneeeeeesnseceeeeneeeeeeneeeeeeeneeeeeeetenaes 1 
g OUNC i ecescbei pete etebens edb ahi eenithenareeeela yeh tefenet ware shucbevantents renee reaneces tied heetateeehs rere teteheataal 1 
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6. Ijoined 4-H because... (Select ALL that apply.) 
a. Of the types of activities that yOU Get tO GO ........eececceeeeeeeeeeeeceeeeeceaeeesaaeseeeeeseaeeesaeeeeaeeeenees 1 
b. My- Trends Were In its... csec tials ect daateeceis cee betel caeeeety etande dinate etnies aededcte ntecdeieadentetis 1 
C. My parents Signed Me Up FOF it... eee eeeeeeeeeeeeeeeeeeeeeeeeeeeeeaeeeeeseaeeeeeseaeeeeeseaeeeeeeeeeeeeeseeeaeees 1 
d. OOO Rita ccc n td ce ceces Gibcu sah sun cesbads atecataieatte siudateds hea Veteaddas tesa vte eedutea Ba vensaci tas teeaeonte che aces beatae 1 
7. Please tell us how much you agree or disagree that this 4-H program or project gives you the 
opportunity to do each of the following things. (Select ONE in each row.) 
Strongly F Strongly 
In this 4-H program or project, I can ... Disagree Osegiee AGIES Agree 
a. Do experiments 1 2 3 4 
b. Do hands-on science activities 1 2 3 4 
c. Solve problems 1 2 3 4 
d. See science in a fun way 1 2 3 4 
e. Learn about careers 1 2 3 4 
f. Serve my community 1 2 3 4 
g. Learn with my friends 1 2 3 4 
h. Get answers to my questions from leaders 1 2 3 4 
i. Tell a group of people about something | 
1 2 3 4 
learned or made 
8. Pick the three things that you like best about coming to this 4-H program or project: (Select only 
THREE.) 
The adults are caring and kind (staff, leaders, VOIUNteerS) ............ccceceeeeeceeececeeeeeeeeeeeaeeeeeeeeteeesnnaeees 1 
I like the CurriculUM/projeCt DOOK ....... ee ce eeeeeeee scene ee eeeeeeeeeeeeaeeeeeeeaeeeseeeaeeeseeaaeeeeeeaaeeeeeeaaeeeeeeaeeeeseaaes 1 
It is a place where | feel Safe... eeece ce ceenceeeeeeeceeeeeeeeeeeesaeeeeeesaeeeeeesaaeeeeesneeeeeeeeeeenenseeeeseeeeeeenenees 1 
It is a group where | feel like | DElONG 20.0... cece eeeeeeee ee eeee eter eeeeaeeeeeeaaeeeeeeaaeeeeeeaaeeeeseaaeeeeeeaaeeeeeeaaes 1 
| can use tools and materials here that | don’t have at school or at NOME ..............eeeeeeeeeeeeeeeeeeeeeeee 1 


om sea*oaooD 


| get to spend time with my fri 
| get to do hands-on activities 


| get to do community service 


va] nfo [oer eee Ere Pr CRE Rr rE CEEETRETEE Er rrrTTere Ter ErrrPETEreRTTe rerrererre errr rate 
ANC PLOJSCIS sik peer seth es Ri eheseai tise piesuudlias eal Peeples 


| get positive feedback from the adults and other kids 


| get opportunities to demonstrate things | have learned or made in front of others 


9. Please tell us how often you think the following things are true when you are at this 4-H program or 
project. (Select ONE in each row.) 


In this 4-H program or project ... Never Sometimes Usually Always 
a. | feel safe and respected 1 2 3 4 
b. lam afraid | will be embarrassed or put down 1 2 3 4 
c. All kinds of kids are welcome 1 2 3 4 
d. Adults listen to what | have to say 1 2 3 4 
e. | feel comfortable going to adults for advice 1 2 3 4 
f. Other kids care about me 1 2 3 4 
g. | feel like | can make a difference 1 2 3 4 
h. lam encouraged to take responsibility 1 2 3 4 
i. It is OK to make mistakes 1 2 3 4 
10. We are interested in how often you do the following things. (Select ONE in each row.) 

When | have a decision to make... Never Sometimes Usually Always 
a. | look for information to help me understand { D 3 
the problem 
b. | think before making a choice 1 2 3 
c. | consider the risks of a choice before making { 9 3 
a decision 
d. | think about all the information | have about { 2 3 
the different choices 
e. | think of past choices when making new 1 2 3 
decisions 
When | think about things... Never Sometimes Usually Always 
a. |can easily express my thoughts ona { 2 3 

problem 
b. | usually have more than one source of 

; se 1 2 3 

information before making a decision 
c. | compare ideas when thinking about a topic 1 2 3 
d. |keep my mind open to different ideas when { 9 3 

planning to make a decision 
e. |am able to tell the best way of handing a { 9 3 

problem 


When solving a problem.... Never Sometimes Usually Always 
a. | first figure out exactly what the problem is 1 2 3 4 
b. I try to determine what caused it 1 2 3 4 
c. |do what! have done in the past to solve it 1 2 3 4 
d. |compare each possible solution with others 
é 1 2 3 4 
to find the best one 
e. After selecting a solution, | think about it for 
; pote : 1 2 3 4 
a while before putting it into action 
f. Once | have solved a problem, | think about { 2 3 4 


how my solution worked 


11. Please indicate the extent to which you agree or disagree with each of the following statements. 
(Select ONE in each row.) (Noyce Enthusiasm for Science Items) 


Strongly ; Strongly 
Disagree Disagree Agree Agree 

a. Science is something | get excited 1 2 3 4 
about 

b. | like to take things apart to learn more 1 2 3 4 
about them 
| like to participate in science projects 1 2 3 4 
I'd like to get a science kit as a gift (for 
example, a microscope, magnifying 1 2 3 4 
glass, a robot, etc.) 

e. | like to see how things are made (for 
example, ice-cream, a TV, an iPhone, 1 2 3 4 
energy, etc) 

f. [like to watch programs on TV about 1 2 3 4 
nature and discoveries 

g. lam curious to learn more about 1 2 3 4 
science, computers or technology 

h. | like to work on science activities 1 2 3 A 

i. When | grow up and have kids, | will { 2 3 4 
take them to a science museum 
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j. | want to understand science (for 


example, to know how computers work, 1 2 3 4 
how rain forms, or how airplanes fly) 

k. lenjoy visiting science museums or 1 2 3 4 
ZOOS 

l. | get excited learning about new { D) 3 4 
discoveries or inventions 

m. | like reading science magazines 1 2 3 4 

n. |pay attention when people talk about 1 9 3 4 
recycling to protect our environment 

o. lam curious to learn more about cars 1 D) 3 4 
that run on electricity 

p. | get excited to find out that | will be 1 D) 3 4 
doing a science activity 

1 2 3 4 


q. | enjoy reading science fiction books 


12. Please indicate the degree to which you agree or disagree with the following statements. (Select 


ONE in each row.) 


Disagree Not Sure Agree 
a. |like science (Noyce item) 1 2 3 
b. |am good at science 1 2 3 
c. Science is boring (Noyce item) 1 2 3 
d. Science is useful for solving everyday problems 1 2 3 
13. Please indicate the extent to which you agree or disagree with the following statements. (Select ONE 
in each row.) 
Strongly ; Strongly 
Disagree Pisagle? mgiee Agree 
a. When | graduate from high school, | would like to 
1 2 3 4 
have a job related to science 
b. Science is one of my favorite subjects 1 2 3 4 
c. Ido science-related activities that are not for 
1 2 3 4 
schoolwork 
d. |take science only because | have to 1 2 3 4 
e. | take science only because it will help me in the { 2 3 4 


future 


14. In the past year, have you done any of the following things? (Select YES or NO.) 


Yes No 
a. Helped with a community service project that relates to science (for { 2 
example: planted trees or gardens, road or stream clean-up, recycling) 
b. Used science tools to help the community (for example: mapped with 1 9 
GIS, tested water quality) 
c. Taught others about science (for example: demonstrated, gave { 2 
presentation at community meeting or at school) 
d. Organized or led science-related events (for example: science fair, { 9 
environmental fair) 
15. How old are you? 
16. Are you... (Select ONE.) 
a. PIDOY oe 82. eather ho, Weave eee eee a8 eo Hohe ae 1a Soe ae, Waker ie es le pel 1 
b PSU seed Beet oc eee iat hs heads Mace tned cepts tc etined Atead seat ated ee ads tea tee danas: ibeiee de a tapes 2 
17. Are you... (Select ALL that apply.) 
a. AtmicansAmerican/Black,, cise ccteracecvees, Soinne ag coevsaeh cp uhtecvent adh Soh. needewestaah sau patveveeesia. Sans ay deere 1 
b SIAM ec 28. secede tevin hecc ni ctesien 2 Meet Maieah Shee ic ccvenmese Acautt Ay eavres, Aten ddecyeesua es Mbanad My cael Mats bc ceyeates 1 
c Hispanic/Latino ceive cde saiin Rien idl ceulineldhiens A iiiteeetd eee ee ene lieiee tid 1 
d. Native American/Alaskan Nattive............:.ccccceccesscceceeeeeeeeeeeaeaeeeeeeeeeeceaeaeeeeeeesesecaaeaeeeeeeeeeeeees 1 
e Native Hawaiian/Other Pacific ISIANGEN..............ccccccecseccecesseeeeeecseeeeeesseeeeeeseneeeesssesaeeeseeeaeens 1 
f Whiteccuntathnei a cihnsl action dian ah diate dae dianlea asin 1 
g OUNGf vise uegeseetastdp Miedociteen diaededheedtd getedoniede tien atug needed g Miele die dead pene dies 1 
18. What type of school do you go to? (Select ONE.) 
a. PubliC: SGHOOM zac dec ss eteshin ented ahead Maile eee eee! 1 
b. Private-SCHOOMc.ccis2) ested aghveliec des dheleeedieded dala weed ete ee 2 
Cc. Religious School (Catholic, tC.) .........ccccccecceeseeeceeeeeceeeeeeaeeeeneeeeeeeecaesesaaeseeeeeseeessnaeeneeeeeaes 3 
d. Flome:SChooli..tacccy adiacit ae cetnieileee Ata Gretel te ental atten Grade ett tateieety ates 4 
19. How far do you want to go in school? (Select ONE.) 
a. Graduate ‘Trom:high: School. :.s0.sc¢ar. chine ea tiah a dine ee a 1 
b. Go to a trade or vocational SCHOOL ...........:ccccesceceseeseeceseeeeeeeeeeeaeeeceeaeeesecaeeesecaeeeseesaeeseseaaes 2 
C. Go to college for a little While ........ cece cecceceeceeeeeeeeeseeeaeeeceeeaeeeseesaeesesesaeeesscaeeeeseneeseseaaes 3 
d Finish:college:.i. cher tate ani ahaa enon tat 4 
e Get more education after COlC GE ...........cccceceessececseseceseeeeeeeseeeaeeeseesaeeessesaeeesseaeeeeseseeeeeeaaes 5 
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20. What kind of work do you expect to be doing when you are 30 years old? (Select ONE.) 


a. FAME TANCM Ol eis, teviatasvseviginutacatdsuesuldavateuaddeutdanadedeundaeawessatatsand awevsvaedeludMarpiavaate ase 1 
b. Military, police, Or SECUrIty OFFICE... eee ee ceeeee ee eeeeeeeeeeeeaeeeeeeaaeeeeeeaaeeeeeeaeeeeeeaas 2 
Cc. Professional buSINESS PErSON OF MANAGET .........:ccceeeeeeceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeaees 3 
d. OWI: DUSINGSS si )ee co esses s caite tears tie rene cag text tpt hinidhg Seah re tgp de Teta eedee ie eee 4 
e. Work in computers or teChnOlOQy ...........cccceeeeceeceeeeeeeeeeeeeaeeeeeeaeeeeeeaaeeeeeenaeeeeeenaeeeeee 5 
f. Scientist OF reSCArChe r .........ceeececececeeeeeceeececeeeeeeeaeeeeaaeeeeeeeceaeeeeaaeseeneeseeeesaesseeeeeeeeess 6 
g. Work in the medical field (doctor, nurse, lab technician) ...........:eeeeeeeeeeeeeeeeneeeeeees 7 
h. Wi ACM aces cise Fevers the the ce ven ag oa Satara 2 ea ea staat ce den Fae ean dt ste teteats datent Mateeen sland eeeveaatelae 8 
i. Artist (writer, dancer, Painter)...........cccceescsceeeesceeeeeeeceeeeeeeaeeeeeeaaeeeseeeaeeeseeaaeeeeeeaaeeneees 9 
j. Skilled craftsperson (Carpenter, DIUMDEL) ............cceeceeeseeeeeee cent eeeaeeeeaeeseeeeeseaeeesaeeseneeeeaes 10 
k. Retail (WOrkKs INA StOMG) 2 so sce26) Socue cd sbigwn eh enece yea ay uclde a cusae aca ipieeh stgeee atanyaeee igsee ths syicere tees aeaged 11 
I. ENGIMGOH OF ANCMILOCK sac c2i5s ietees idee ad seseeee hogeae adasaccuts ius ae atd suitets segees seagate ties ae ec teueete tegeessedtad 12 
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If you are 12 years old or younger, please answer this question. If you are 13 or older, 
please skip this question and go on to the next question. 
21a. Please let us know whether each of these statements is true for you. (Select YES or NO.) 
Yes No 
a. |cando an experiment to answer a question 1 2 
b. | can tell others how to do an experiment 1 2 
c. [can write down information correctly 1 2 
d. I|can make achart or picture to show information 1 2 
e. |can explain why things happen in an experiment 1 2 


If you are 12 or younger, you have finished the survey. Thank you! 
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If you are 13 or older, please answer this question. 


21b. Please let us know how often each of these statements is true for you. (Select ONE in each row.) 


Never Sometimes Usually Always 
a. [can use scientific knowledge to form a question 1 2 3 4 
b. |can ask a question that can be answered by 
1 2 3 4 
collecting data 
c. Ican design a scientific procedure to answer a { 9 3 4 
question 
d. |can communicate a scientific procedure to others 1 2 3 4 
e. |can record data accurately 1 2 3 4 
f. | can use data to create a graph for presentation to 
1 2 3 4 
others 
g. Icancreate a display to communicate my data and 1 2 3 4 
observations 
h. [can analyze the results of a scientific investigation 1 2 3 4 
i. | can use science terms to share my results 1 2 3 4 
j.|can use models to explain my results 1 2 3 4 
k. lcan use the results of my investigation to answer { 9 3 4 
the questions | asked 


Thank you! 


Appendix C 
Statistical Properties of Survey Scales 


For each series of survey items addressing a common theme, evaluators created a survey 
scale to measure Science participants’ overall response to that theme. This appendix describes 
the individual items that are included in each scale, and presents the following statistical 
properties of each scale: 


7 Cronbach’s Alpha: a measure of the internal consistency of the survey scale 
ranging from 0-1, with higher numbers indicating a greater cohesiveness of items 


7 Mean: the average score on the scale across all participants, ranging from 1 to 4 
7 Standard deviation: an estimate of the average variability of the scale data 

7 Minimum/maximum: the minimum and maximum scores possible on the scale 
: 25" percentile/75" percentile: respectively, the scale scores below which 25 


percent of participants and 75 percent of participants scored 
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Critical Thinking Scale 


The critical thinking scale was computed to range from one to four, with four indicating that on 
average, participants said they always do the following: 


When I think about things I... 


Can easily express my thoughts on a problem 

Usually have more than one source of information before making a decision 
Compare ideas when thinking about a topic 

Keep my mind open to different ideas when planning to make a decision 
Am able to tell the best way of handling a problem 


Items adapted from: Perkins & Mincemoyer, 2002. 


Statistical Properties: 


Alpha Mean Standard Minimum 25° i os Maximum 
Deviation Percentile | Percentile 
0.76 | 294 | 063 | 1 | 260 | 340 | 4 
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Decision Making 


The Decision Making Scale was computed to range from one to four, with four indicating that on 
average participants said they always do the following: 


When I have a decision to make I... 


Look for information to help me understand the problem 

Think before making a choice 

Consider the risks of a choice before making a decision 

Think about all the information I have about the different choices 
Think of past choices when making new decisions 


Items adapted from: Perkins & Mincemoyer, 2002. 


Statistical Properties: 


Alpha Mean Standard Minimum 25° 4 Maximum 
Deviation Percentile | Percentile 
0.77 | 306 | 064 | 1 | 260 | 360 | 4 
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Problem Solving 


The Problem Solving Scale was computed to range from one to four, with four indicating that on 
average participants said they always do the following: 


When solving a problem... 


I first figure out exactly what the problem is 

I try to determine what caused it 

I do what I have done in the past to solve it 

I compare each possible solution with others to find the best one 

After selecting a solution, I think about it for a while before putting it into action 
Once I have solved a problem, I think about how my solution worked 


Items adapted from: Perkins & Mincemoyer, 2002. 


Statistical Properties: 


Alpha Mean Standard Minimum 25" cia Maximum 
Deviation Percentile | Percentile 
0.74 | 294 | 060 | 1 | 250 | 3.33 | 4 
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Noyce Enthusiasm Scale 


The Noyce Enthusiasm Scale was computed to range from one to four, with four indicating that 
participants strongly agreed with the following statements: 


m Science is something I get excited about = | want to understand science (for example, 
m [like to take things apart to learn more to know how computers work, how rain 
about them forms, or how airplanes fly) 
m [like to participate in science projects m [enjoy visiting science museums or Zoos 
m Id like to get a science kit as a gift (for m I get excited learning about new 
example, a microscope, magnifying glass, discoveries or inventions 
a robot, etc.) m [like reading science magazines 
m | like to see how things are made (for m I pay attention when people talk about 
example, ice-cream, a TV, an iPhone, recycling to protect our environment 
energy, etc) m= lamcurious to learn more about cars that 
m= lamcurious to learn more about science, run on electricity 
computers or technology m= | get excited to find out that I will be doing 
m [like to watch programs on TV about a science activity 
nature and discoveries m I enjoy reading science fiction books 
m [like to work on science activities m [like science 
m When I grow up and have kids, I will take m Science is boring (item was reverse-coded) 
them to a science museum 
Alpha | Mean | Standard | Minimum | (cla | DD | Maximum 
Deviation Percentile | Percentile 
091 | 293 | 055 | 1 | 256 | 3.33 | 4 
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Science Process Skills Inventory, Youth Ages 13 to 18 


The Science Process Skills Inventory was computed to range from one to four, with four 
indicating that on average, participants said the following statements are always true: 


I can use scientific knowledge to form a question 

I can ask a question that can be answered by collecting data 

I can design a scientific procedure to answer a question 

I can communicate a scientific procedure to others 

I can record data accurately 

I can use data to create a graph for presentation to others 

I can create a display to communicate my data and observations 
I can analyze the results of a scientific investigation 

I can use science terms to share my results 

I can use models to explain my results 

I can use the results of my investigation to answer the questions I asked 


These questions were only asked of youth ages 13 and older. 


Items adapted from: Arnold & Bordeau, 2009. 


Alpha Mean Standard Minimum 25° Phin Maximum 
Deviation Percentile | Percentile 
0.91 298 | 059 | 173 | 255 | 34 | 4 
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Program Benefits & Opportunities Scale 


The Program Benefits and Opportunities scale was computed to range from one to four, with 
four indicating that on average, participants strongly agree that they can do the following in their 
4-H program or project: 


In this 4-H program or project, I can... 


Do experiments 

Do hands-on science activities 

Solve problems 

See science in a fun way 

Learn about careers 

Serve my community 

Learn with my friends 

Get answers to my questions from leaders 

Tell a group of people about something I learned or made 


Alpha Mean Standard Minimum 25" pow Maximum 
Deviation Percentile | Percentile 


082 | 3.10 | O51 | 1 | 2.78 | 344 | 4 
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Program Climate Scale 


The Program Climate scale was computed to range from one to four, with four indicating that on 
average, participants said the following things are always true about their program or project: 


In this 4-H program or project... 


I feel safe and respected 

I am afraid I will be embarrassed or put down (item was reverse-coded) 
All kinds of kids are welcome 

Adults listen to what I have to say 

I feel comfortable going to adults for advice 

Other kids care about me 

I feel like I can make a difference 

I am encouraged to take responsibility 

It is OK to make mistakes 


Items adapted from: Silliman, 2008. 


Alpha Mean Standard Minimum a5" pow Maximum 
Deviation Percentile | Percentile 
0.75 | 3 | o51 | 133 | 267 | 341 | 4 
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Community Science Scale 


The Community Science Scale was computed to range from one to four, with four indicating that 
youth participated in all of the following activities within the past year: 


a Helped with a community service project that relates to science (for example: 
planted trees or gardens, road or stream clean-up, recycling) 

a Used science tools to help the community (for example: mapped with GIS, tested 
water quality) 

5 Taught others about science (for example: demonstrated, gave presentation at 
community meeting or at school) 

a Organized or led science-related events (for example: science fair, environmental 
fair) 


Items adapted from: Silliman, 2010. 


Alpha Mean Standard Minimum 25° 7S Maximum 
Deviation Percentile | Percentile 
0.64 | 182 | 127 | 0 | 1 | 3 | 4 
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Appendix D 
2005 and 2009 National Assessment of Educational Progress 
Survey Items Used on the YEAK Survey 


How much do you agree 
or disagree with the 
following statements? 


| like science 
| am good at science 


Science is boring 


Science is useful for 
solving everyday 
problems 


Science is one of my 
favorite subjects 


| do science-related 
activities that are not for 
schoolwork 


| take science only 
because | have to 


| take science only 
because it will help me in 
the future. 


When | graduate from 
high school, | would like to 
have a job related to 
science. 


Exhibit reads: In 2005, the item “I like science” was asked to fourth-, eighth-, and twelfth-grade youth surveyed in the 
NAEP Science assessment. The item was not asked at any grade in 2009. 
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Appendix E 


Narrative Comparison of 2010 and 2011 YEAK Survey Data 


Overview 


The comparisons detailed here examine the differences and similarities between youth 
responses on the 2010 and 2011 YEAK surveys. Each comparison is a general statement about 
the overall trends in the two datasets; no statistical tests were performed to assess any differences 
between the two samples because we used different sampling methods in 2010 and 2011. 


Demographics 


Youth in the 2010 and 2011 YEAK samples hailed from similar racial and ethnic 
backgrounds. Roughly equal numbers of males and females participated in each 
survey, but the 2011 sample had slightly more younger youth (ages 9 to 12) than 
the 2010 sample. Fewer youth in the 2011 sample attended public school. 


Youth Experiences in 4-H & 4-H Science 


In 2011, a slightly higher percentage of youth reported being in 4-H for three 
years or more. More youth in the 2011 sample had been in their projects for eight 
months or more, and these youth also reported spending more time each week on 
their projects. 


Program Benefits & Climate 


Life Skills 


Youth responded similarly on their reasons for joining 4-H. Youth in each of the 
two samples also agreed on the opportunities they have in their science projects, 
favorite characteristics of their programs, and overall program climate. 


In general, youth gave similar accounts of their problem solving, critical thinking, 
and decision making skills. 


Science Process Skills 


Among respondents ages 9 to 12, youth in the two samples reported similar 
science process skills. Generally, the same was true among youth ages 13 to 18. 
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However, more youth in the 2011 sample reported that they can always or usually 
use the results of an investigation to answer the questions they asked. 


Attitudes & Enthusiasm toward Science 


Youth in each of the two samples shared similar attitudes and enthusiasm about 
science. 


Academic Aspirations & Interest in Science Career 


More youth in the 2011 sample want to pursue more education after college than 
youth in the 2010 sample. Roughly similar numbers of youth said they want to 
have a science-related career after graduating from high school. 


Comparisons Based on Youth Characteristics 


Race and ethnicity. We cannot compare findings between 2010 and 2011 in 
regards to race and ethnicity because we used different methods to categorize 
youth in each year. In 2010 we analyzed the differences between white and 
African American youth; in 2011 we analyzed the differences between youth 
whose race or ethnicity is well-represented in science fields and youth whose race 
or ethnicity is underrepresented in science fields. 


Gender. Analyses of 2010 youth survey data revealed that a significantly larger 
percentage of boys reported wanting to pursue a science career than did girls. In 
2011 there was no significant difference between the career aspirations of boys 
and girls. In 2011 boys were more positive than girls on a scale measuring their 
enthusiasm toward science, but girls gave higher ratings of their own decision 
making and critical thinking skills than did boys. In 2011 girls also gave higher 
average ratings to their program’s overall climate. 


Exposure to 4-H Science. In both 2010 and 2011 we found that youth with 
higher levels of exposure to 4-H Science programming gave higher average 
ratings of their own life skills than did youth in the low-exposure group. 2010 
high-exposure respondents gave a more positive rating of their program 
environment than did low-exposure youth; in 2011 high-exposure youth gave 
more positive assessments of the benefits they received from participating in the 
program. 
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